Computer Vision: MNIST revisted as a CNN

In this example, we are going to revisit the MNIST data set but use a CNN to classify the digits. This will take us one step closer to image classification and you will learn the fundamental concepts behind CNNs. ℹ️

Learning objectives:

• Why is translation invariant & spatial hierarchy important
• What the general structure of CNN models looks like
• What is the convolution operation
• What are feature maps
• How pooling helps by downsampling

Required packages

library(keras)

Prepare data

Let’s import our training and test data. Rather than turn our image data into a 2D tensor as we did in the earlier module, here we convert our data to a 4D tensor that has:

• 60K samples (train) and 10K samples (test)
• height x width = 28x28 pixels
• 1 color channel (these are gray scale rather than RGB, which has 3 color channels)

As before, our pixels range from 0-255 so we re-scale them to be between 0-1.

mnist <- dataset_mnist()
c(c(train_images, train_labels), c(test_images, test_labels)) %<-% mnist

train_images <- array_reshape(train_images, c(60000, 28, 28, 1)) / 255
test_images <- array_reshape(test_images, c(10000, 28, 28, 1)) / 255

train_labels <- to_categorical(train_labels)
test_labels <- to_categorical(test_labels)

CNN: Feature detector

To run a CNN we will follow a very similar approach to what we’ve seen so far. The main difference is that we create a convolution and max pooling procedure prior to our densley connected MLP. This is known as our feature detector step.

We’ll discuss the details of these steps shortly but for now just realize these main points:

1. our input_shape is 28x28 image with 1 color channel,
2. the output of each layer_conv2d() and layer_max_pooling_2d() is a 3D tensor of shape (height, width, channels),
3. the height and width dimensions tend to shrink as you go deeper in the network,
4. while the number of channels increase.

filter_size <- c(3L, 3L)
padding_selection <- "same"
model <- keras_model_sequential() %>%
  
  layer_conv_2d(filters = 32, kernel_size = filter_size, activation = "relu", 
                padding = padding_selection, input_shape = c(28, 28, 1)) %>%
  layer_max_pooling_2d(pool_size = c(2, 2)) %>%
  
  layer_conv_2d(filters = 64, kernel_size = filter_size, activation = "relu", 
                padding = padding_selection) %>%
  layer_max_pooling_2d(pool_size = c(2, 2)) %>%
  
  layer_conv_2d(filters = 64, kernel_size = filter_size, activation = "relu", 
                padding = padding_selection)

summary(model)

Model: "sequential_14"
________________________________________________________________________________
Layer (type)                        Output Shape                    Param #     
================================================================================
conv2d_42 (Conv2D)                  (None, 28, 28, 32)              320         
________________________________________________________________________________
max_pooling2d_29 (MaxPooling2D)     (None, 14, 14, 32)              0           
________________________________________________________________________________
conv2d_41 (Conv2D)                  (None, 14, 14, 64)              18496       
________________________________________________________________________________
max_pooling2d_28 (MaxPooling2D)     (None, 7, 7, 64)                0           
________________________________________________________________________________
conv2d_40 (Conv2D)                  (None, 7, 7, 64)                36928       
================================================================================
Total params: 55,744
Trainable params: 55,744
Non-trainable params: 0
________________________________________________________________________________

CNN: Classifier

Next, we feed the last output tensor of shape (3, 3, 64) into a densely connected MLP. This MLP is to classify our images and we often refer to this part of our CNN as the classifier. The only new concept here is layer_flatten() which is reducing the 3D tensor for a given image to a 1D tensor.

model %>%
  layer_flatten() %>%
  layer_dense(units = 64, activation = "relu") %>%
  layer_dense(units = 10, activation = "softmax")

summary(model)

Model: "sequential"
________________________________________________________________________________________________
Layer (type)                               Output Shape                          Param #        
================================================================================================
conv2d_2 (Conv2D)                          (None, 26, 26, 32)                    320            
________________________________________________________________________________________________
max_pooling2d_1 (MaxPooling2D)             (None, 13, 13, 32)                    0              
________________________________________________________________________________________________
conv2d_1 (Conv2D)                          (None, 11, 11, 64)                    18496          
________________________________________________________________________________________________
max_pooling2d (MaxPooling2D)               (None, 5, 5, 64)                      0              
________________________________________________________________________________________________
conv2d (Conv2D)                            (None, 3, 3, 64)                      36928          
________________________________________________________________________________________________
flatten (Flatten)                          (None, 576)                           0              
________________________________________________________________________________________________
dense_1 (Dense)                            (None, 64)                            36928          
________________________________________________________________________________________________
dense (Dense)                              (None, 10)                            650            
================================================================================================
Total params: 93,322
Trainable params: 93,322
Non-trainable params: 0
________________________________________________________________________________________________

CNN: Compile & train

These steps are the same as before. However, you will notice that training takes longer, which is due to the added CNN procedure. While this model is training, let’s discuss what’s happening under the hood of a CNN ℹ️.

Although we used a fairly basic model without optimizing the learning rate, model capacity, batch, etc., you will also notice that our model performance is superior to our MLP model from the earlier module:

• MLP: loss (~ 0.07) & accuracy (~ 0.975)
• CNN: loss (~ 0.04) & accuracy (~ 0.99)

model %>% compile(
  optimizer = "rmsprop",
  loss = "categorical_crossentropy",
  metrics = c("accuracy")
)

history <- model %>% fit(
  train_images, train_labels,
  epochs = 5, 
  batch_size = 128,
  validation_split = 0.2
)

2021-11-04 05:34:05.294206: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)

Epoch 1/5

  1/375 [..............................] - ETA: 7:35 - loss: 2.3010 - accuracy: 0.0703
  4/375 [..............................] - ETA: 14s - loss: 2.2099 - accuracy: 0.2402 
  5/375 [..............................] - ETA: 16s - loss: 2.1407 - accuracy: 0.2984
  6/375 [..............................] - ETA: 17s - loss: 2.0954 - accuracy: 0.3073
  7/375 [..............................] - ETA: 18s - loss: 2.0518 - accuracy: 0.3170
  8/375 [..............................] - ETA: 18s - loss: 1.9982 - accuracy: 0.3320
 10/375 [..............................] - ETA: 17s - loss: 1.8878 - accuracy: 0.3695
 12/375 [..............................] - ETA: 17s - loss: 1.7937 - accuracy: 0.3945
 14/375 [>.............................] - ETA: 17s - loss: 1.7051 - accuracy: 0.4336
 15/375 [>.............................] - ETA: 17s - loss: 1.6531 - accuracy: 0.4568
 16/375 [>.............................] - ETA: 17s - loss: 1.6055 - accuracy: 0.4766
 18/375 [>.............................] - ETA: 17s - loss: 1.5476 - accuracy: 0.4983
 19/375 [>.............................] - ETA: 17s - loss: 1.5085 - accuracy: 0.5127
 20/375 [>.............................] - ETA: 17s - loss: 1.4708 - accuracy: 0.5270
 21/375 [>.............................] - ETA: 17s - loss: 1.4323 - accuracy: 0.5435
 23/375 [>.............................] - ETA: 17s - loss: 1.3591 - accuracy: 0.5662
 25/375 [=>............................] - ETA: 17s - loss: 1.3259 - accuracy: 0.5756
 26/375 [=>............................] - ETA: 17s - loss: 1.3035 - accuracy: 0.5841
 28/375 [=>............................] - ETA: 17s - loss: 1.2592 - accuracy: 0.5993
 30/375 [=>............................] - ETA: 17s - loss: 1.2086 - accuracy: 0.6177
 32/375 [=>............................] - ETA: 16s - loss: 1.1601 - accuracy: 0.6338
 34/375 [=>............................] - ETA: 16s - loss: 1.1233 - accuracy: 0.6443
 36/375 [=>............................] - ETA: 16s - loss: 1.0899 - accuracy: 0.6539
 38/375 [==>...........................] - ETA: 16s - loss: 1.0626 - accuracy: 0.6606
 40/375 [==>...........................] - ETA: 16s - loss: 1.0385 - accuracy: 0.6670
 42/375 [==>...........................] - ETA: 15s - loss: 1.0066 - accuracy: 0.6782
 44/375 [==>...........................] - ETA: 15s - loss: 0.9828 - accuracy: 0.6857
 46/375 [==>...........................] - ETA: 15s - loss: 0.9565 - accuracy: 0.6945
 48/375 [==>...........................] - ETA: 15s - loss: 0.9324 - accuracy: 0.7028
 50/375 [===>..........................] - ETA: 15s - loss: 0.9136 - accuracy: 0.7089
 52/375 [===>..........................] - ETA: 15s - loss: 0.8929 - accuracy: 0.7150
 54/375 [===>..........................] - ETA: 15s - loss: 0.8763 - accuracy: 0.7205
 56/375 [===>..........................] - ETA: 15s - loss: 0.8604 - accuracy: 0.7252
 58/375 [===>..........................] - ETA: 14s - loss: 0.8418 - accuracy: 0.7314
 59/375 [===>..........................] - ETA: 14s - loss: 0.8337 - accuracy: 0.7342
 61/375 [===>..........................] - ETA: 14s - loss: 0.8192 - accuracy: 0.7391
 63/375 [====>.........................] - ETA: 14s - loss: 0.8075 - accuracy: 0.7422
 65/375 [====>.........................] - ETA: 14s - loss: 0.7933 - accuracy: 0.7478
 67/375 [====>.........................] - ETA: 14s - loss: 0.7803 - accuracy: 0.7521
 69/375 [====>.........................] - ETA: 14s - loss: 0.7681 - accuracy: 0.7557
 71/375 [====>.........................] - ETA: 14s - loss: 0.7547 - accuracy: 0.7600
 73/375 [====>.........................] - ETA: 14s - loss: 0.7410 - accuracy: 0.7642
 75/375 [=====>........................] - ETA: 14s - loss: 0.7314 - accuracy: 0.7673
 77/375 [=====>........................] - ETA: 14s - loss: 0.7229 - accuracy: 0.7699
 79/375 [=====>........................] - ETA: 13s - loss: 0.7132 - accuracy: 0.7727
 81/375 [=====>........................] - ETA: 13s - loss: 0.7040 - accuracy: 0.7761
 83/375 [=====>........................] - ETA: 13s - loss: 0.6941 - accuracy: 0.7792
 85/375 [=====>........................] - ETA: 13s - loss: 0.6829 - accuracy: 0.7826
 87/375 [=====>........................] - ETA: 13s - loss: 0.6749 - accuracy: 0.7853
 89/375 [======>.......................] - ETA: 13s - loss: 0.6661 - accuracy: 0.7881
 91/375 [======>.......................] - ETA: 13s - loss: 0.6579 - accuracy: 0.7910
 93/375 [======>.......................] - ETA: 13s - loss: 0.6508 - accuracy: 0.7936
 95/375 [======>.......................] - ETA: 13s - loss: 0.6426 - accuracy: 0.7962
 97/375 [======>.......................] - ETA: 13s - loss: 0.6334 - accuracy: 0.7989
 99/375 [======>.......................] - ETA: 12s - loss: 0.6256 - accuracy: 0.8015
101/375 [=======>......................] - ETA: 12s - loss: 0.6183 - accuracy: 0.8038
102/375 [=======>......................] - ETA: 12s - loss: 0.6173 - accuracy: 0.8043
103/375 [=======>......................] - ETA: 12s - loss: 0.6137 - accuracy: 0.8056
104/375 [=======>......................] - ETA: 12s - loss: 0.6105 - accuracy: 0.8067
105/375 [=======>......................] - ETA: 12s - loss: 0.6061 - accuracy: 0.8082
107/375 [=======>......................] - ETA: 12s - loss: 0.5977 - accuracy: 0.8109
108/375 [=======>......................] - ETA: 12s - loss: 0.5931 - accuracy: 0.8125
110/375 [=======>......................] - ETA: 12s - loss: 0.5853 - accuracy: 0.8152
112/375 [=======>......................] - ETA: 12s - loss: 0.5794 - accuracy: 0.8169
114/375 [========>.....................] - ETA: 12s - loss: 0.5743 - accuracy: 0.8185
116/375 [========>.....................] - ETA: 12s - loss: 0.5663 - accuracy: 0.8211
118/375 [========>.....................] - ETA: 12s - loss: 0.5598 - accuracy: 0.8234
119/375 [========>.....................] - ETA: 12s - loss: 0.5570 - accuracy: 0.8242
120/375 [========>.....................] - ETA: 12s - loss: 0.5542 - accuracy: 0.8251
121/375 [========>.....................] - ETA: 12s - loss: 0.5506 - accuracy: 0.8263
123/375 [========>.....................] - ETA: 12s - loss: 0.5455 - accuracy: 0.8279
124/375 [========>.....................] - ETA: 12s - loss: 0.5430 - accuracy: 0.8289
125/375 [=========>....................] - ETA: 12s - loss: 0.5405 - accuracy: 0.8297
126/375 [=========>....................] - ETA: 12s - loss: 0.5382 - accuracy: 0.8305
127/375 [=========>....................] - ETA: 12s - loss: 0.5352 - accuracy: 0.8314
128/375 [=========>....................] - ETA: 12s - loss: 0.5323 - accuracy: 0.8323
130/375 [=========>....................] - ETA: 12s - loss: 0.5259 - accuracy: 0.8344
131/375 [=========>....................] - ETA: 12s - loss: 0.5228 - accuracy: 0.8354
132/375 [=========>....................] - ETA: 12s - loss: 0.5207 - accuracy: 0.8361
133/375 [=========>....................] - ETA: 12s - loss: 0.5194 - accuracy: 0.8365
135/375 [=========>....................] - ETA: 11s - loss: 0.5145 - accuracy: 0.8380
136/375 [=========>....................] - ETA: 11s - loss: 0.5122 - accuracy: 0.8388
137/375 [=========>....................] - ETA: 11s - loss: 0.5099 - accuracy: 0.8395
139/375 [==========>...................] - ETA: 11s - loss: 0.5065 - accuracy: 0.8406
141/375 [==========>...................] - ETA: 11s - loss: 0.5017 - accuracy: 0.8422
143/375 [==========>...................] - ETA: 11s - loss: 0.4967 - accuracy: 0.8439
145/375 [==========>...................] - ETA: 11s - loss: 0.4929 - accuracy: 0.8453
147/375 [==========>...................] - ETA: 11s - loss: 0.4887 - accuracy: 0.8467
149/375 [==========>...................] - ETA: 11s - loss: 0.4849 - accuracy: 0.8479
151/375 [===========>..................] - ETA: 11s - loss: 0.4801 - accuracy: 0.8493
153/375 [===========>..................] - ETA: 10s - loss: 0.4757 - accuracy: 0.8506
155/375 [===========>..................] - ETA: 10s - loss: 0.4714 - accuracy: 0.8520
157/375 [===========>..................] - ETA: 10s - loss: 0.4678 - accuracy: 0.8532
159/375 [===========>..................] - ETA: 10s - loss: 0.4653 - accuracy: 0.8541
161/375 [===========>..................] - ETA: 10s - loss: 0.4627 - accuracy: 0.8549
163/375 [============>.................] - ETA: 10s - loss: 0.4601 - accuracy: 0.8557
165/375 [============>.................] - ETA: 10s - loss: 0.4561 - accuracy: 0.8570
167/375 [============>.................] - ETA: 10s - loss: 0.4528 - accuracy: 0.8578
169/375 [============>.................] - ETA: 10s - loss: 0.4495 - accuracy: 0.8590
171/375 [============>.................] - ETA: 9s - loss: 0.4460 - accuracy: 0.8602 
173/375 [============>.................] - ETA: 9s - loss: 0.4419 - accuracy: 0.8615
175/375 [=============>................] - ETA: 9s - loss: 0.4391 - accuracy: 0.8625
177/375 [=============>................] - ETA: 9s - loss: 0.4360 - accuracy: 0.8633
179/375 [=============>................] - ETA: 9s - loss: 0.4336 - accuracy: 0.8639
181/375 [=============>................] - ETA: 9s - loss: 0.4301 - accuracy: 0.8649
183/375 [=============>................] - ETA: 9s - loss: 0.4264 - accuracy: 0.8660
185/375 [=============>................] - ETA: 9s - loss: 0.4240 - accuracy: 0.8668
187/375 [=============>................] - ETA: 9s - loss: 0.4213 - accuracy: 0.8676
189/375 [==============>...............] - ETA: 9s - loss: 0.4187 - accuracy: 0.8683
191/375 [==============>...............] - ETA: 8s - loss: 0.4156 - accuracy: 0.8692
193/375 [==============>...............] - ETA: 8s - loss: 0.4132 - accuracy: 0.8701
195/375 [==============>...............] - ETA: 8s - loss: 0.4099 - accuracy: 0.8712
197/375 [==============>...............] - ETA: 8s - loss: 0.4068 - accuracy: 0.8723
199/375 [==============>...............] - ETA: 8s - loss: 0.4035 - accuracy: 0.8733
201/375 [===============>..............] - ETA: 8s - loss: 0.4005 - accuracy: 0.8741
202/375 [===============>..............] - ETA: 8s - loss: 0.3993 - accuracy: 0.8745
204/375 [===============>..............] - ETA: 8s - loss: 0.3976 - accuracy: 0.8750
206/375 [===============>..............] - ETA: 8s - loss: 0.3951 - accuracy: 0.8758
208/375 [===============>..............] - ETA: 8s - loss: 0.3921 - accuracy: 0.8767
210/375 [===============>..............] - ETA: 7s - loss: 0.3900 - accuracy: 0.8773
212/375 [===============>..............] - ETA: 7s - loss: 0.3871 - accuracy: 0.8782
214/375 [================>.............] - ETA: 7s - loss: 0.3845 - accuracy: 0.8791
215/375 [================>.............] - ETA: 7s - loss: 0.3836 - accuracy: 0.8795
217/375 [================>.............] - ETA: 7s - loss: 0.3810 - accuracy: 0.8802
219/375 [================>.............] - ETA: 7s - loss: 0.3784 - accuracy: 0.8811
221/375 [================>.............] - ETA: 7s - loss: 0.3758 - accuracy: 0.8820
223/375 [================>.............] - ETA: 7s - loss: 0.3737 - accuracy: 0.8825
224/375 [================>.............] - ETA: 7s - loss: 0.3726 - accuracy: 0.8829
226/375 [=================>............] - ETA: 7s - loss: 0.3703 - accuracy: 0.8837
228/375 [=================>............] - ETA: 7s - loss: 0.3684 - accuracy: 0.8844
230/375 [=================>............] - ETA: 6s - loss: 0.3662 - accuracy: 0.8851
232/375 [=================>............] - ETA: 6s - loss: 0.3641 - accuracy: 0.8857
234/375 [=================>............] - ETA: 6s - loss: 0.3620 - accuracy: 0.8864
236/375 [=================>............] - ETA: 6s - loss: 0.3601 - accuracy: 0.8870
237/375 [=================>............] - ETA: 6s - loss: 0.3589 - accuracy: 0.8874
239/375 [==================>...........] - ETA: 6s - loss: 0.3567 - accuracy: 0.8881
241/375 [==================>...........] - ETA: 6s - loss: 0.3544 - accuracy: 0.8888
243/375 [==================>...........] - ETA: 6s - loss: 0.3530 - accuracy: 0.8893
245/375 [==================>...........] - ETA: 6s - loss: 0.3514 - accuracy: 0.8899
247/375 [==================>...........] - ETA: 6s - loss: 0.3495 - accuracy: 0.8905
249/375 [==================>...........] - ETA: 6s - loss: 0.3469 - accuracy: 0.8913
251/375 [===================>..........] - ETA: 5s - loss: 0.3450 - accuracy: 0.8919
253/375 [===================>..........] - ETA: 5s - loss: 0.3429 - accuracy: 0.8925
255/375 [===================>..........] - ETA: 5s - loss: 0.3414 - accuracy: 0.8930
257/375 [===================>..........] - ETA: 5s - loss: 0.3397 - accuracy: 0.8935
259/375 [===================>..........] - ETA: 5s - loss: 0.3383 - accuracy: 0.8940
261/375 [===================>..........] - ETA: 5s - loss: 0.3362 - accuracy: 0.8948
263/375 [====================>.........] - ETA: 5s - loss: 0.3344 - accuracy: 0.8953
265/375 [====================>.........] - ETA: 5s - loss: 0.3330 - accuracy: 0.8958
267/375 [====================>.........] - ETA: 5s - loss: 0.3318 - accuracy: 0.8963
269/375 [====================>.........] - ETA: 5s - loss: 0.3303 - accuracy: 0.8967
270/375 [====================>.........] - ETA: 5s - loss: 0.3293 - accuracy: 0.8970
271/375 [====================>.........] - ETA: 5s - loss: 0.3284 - accuracy: 0.8973
272/375 [====================>.........] - ETA: 4s - loss: 0.3276 - accuracy: 0.8975
273/375 [====================>.........] - ETA: 4s - loss: 0.3266 - accuracy: 0.8978
275/375 [=====================>........] - ETA: 4s - loss: 0.3250 - accuracy: 0.8983
277/375 [=====================>........] - ETA: 4s - loss: 0.3240 - accuracy: 0.8986
278/375 [=====================>........] - ETA: 4s - loss: 0.3229 - accuracy: 0.8989
279/375 [=====================>........] - ETA: 4s - loss: 0.3221 - accuracy: 0.8992
281/375 [=====================>........] - ETA: 4s - loss: 0.3204 - accuracy: 0.8998
282/375 [=====================>........] - ETA: 4s - loss: 0.3195 - accuracy: 0.9000
284/375 [=====================>........] - ETA: 4s - loss: 0.3181 - accuracy: 0.9004
285/375 [=====================>........] - ETA: 4s - loss: 0.3173 - accuracy: 0.9006
286/375 [=====================>........] - ETA: 4s - loss: 0.3165 - accuracy: 0.9008
288/375 [======================>.......] - ETA: 4s - loss: 0.3152 - accuracy: 0.9013
290/375 [======================>.......] - ETA: 4s - loss: 0.3143 - accuracy: 0.9016
292/375 [======================>.......] - ETA: 4s - loss: 0.3127 - accuracy: 0.9021
294/375 [======================>.......] - ETA: 3s - loss: 0.3110 - accuracy: 0.9025
295/375 [======================>.......] - ETA: 3s - loss: 0.3104 - accuracy: 0.9028
296/375 [======================>.......] - ETA: 3s - loss: 0.3094 - accuracy: 0.9031
298/375 [======================>.......] - ETA: 3s - loss: 0.3078 - accuracy: 0.9036
300/375 [=======================>......] - ETA: 3s - loss: 0.3067 - accuracy: 0.9039
302/375 [=======================>......] - ETA: 3s - loss: 0.3056 - accuracy: 0.9042
304/375 [=======================>......] - ETA: 3s - loss: 0.3041 - accuracy: 0.9047
306/375 [=======================>......] - ETA: 3s - loss: 0.3029 - accuracy: 0.9052
308/375 [=======================>......] - ETA: 3s - loss: 0.3014 - accuracy: 0.9056
310/375 [=======================>......] - ETA: 3s - loss: 0.2999 - accuracy: 0.9060
311/375 [=======================>......] - ETA: 3s - loss: 0.2996 - accuracy: 0.9061
313/375 [========================>.....] - ETA: 3s - loss: 0.2982 - accuracy: 0.9066
315/375 [========================>.....] - ETA: 2s - loss: 0.2970 - accuracy: 0.9069
317/375 [========================>.....] - ETA: 2s - loss: 0.2958 - accuracy: 0.9072
318/375 [========================>.....] - ETA: 2s - loss: 0.2952 - accuracy: 0.9073
319/375 [========================>.....] - ETA: 2s - loss: 0.2950 - accuracy: 0.9075
321/375 [========================>.....] - ETA: 2s - loss: 0.2936 - accuracy: 0.9079
322/375 [========================>.....] - ETA: 2s - loss: 0.2929 - accuracy: 0.9081
324/375 [========================>.....] - ETA: 2s - loss: 0.2913 - accuracy: 0.9087
326/375 [=========================>....] - ETA: 2s - loss: 0.2900 - accuracy: 0.9090
328/375 [=========================>....] - ETA: 2s - loss: 0.2888 - accuracy: 0.9094
329/375 [=========================>....] - ETA: 2s - loss: 0.2880 - accuracy: 0.9097
331/375 [=========================>....] - ETA: 2s - loss: 0.2868 - accuracy: 0.9100
333/375 [=========================>....] - ETA: 2s - loss: 0.2854 - accuracy: 0.9104
334/375 [=========================>....] - ETA: 2s - loss: 0.2850 - accuracy: 0.9105
335/375 [=========================>....] - ETA: 1s - loss: 0.2852 - accuracy: 0.9104
337/375 [=========================>....] - ETA: 1s - loss: 0.2839 - accuracy: 0.9107
338/375 [==========================>...] - ETA: 1s - loss: 0.2833 - accuracy: 0.9109
340/375 [==========================>...] - ETA: 1s - loss: 0.2821 - accuracy: 0.9113
341/375 [==========================>...] - ETA: 1s - loss: 0.2816 - accuracy: 0.9114
343/375 [==========================>...] - ETA: 1s - loss: 0.2806 - accuracy: 0.9117
345/375 [==========================>...] - ETA: 1s - loss: 0.2792 - accuracy: 0.9122
347/375 [==========================>...] - ETA: 1s - loss: 0.2784 - accuracy: 0.9125
349/375 [==========================>...] - ETA: 1s - loss: 0.2771 - accuracy: 0.9129
351/375 [===========================>..] - ETA: 1s - loss: 0.2760 - accuracy: 0.9132
352/375 [===========================>..] - ETA: 1s - loss: 0.2753 - accuracy: 0.9135
353/375 [===========================>..] - ETA: 1s - loss: 0.2748 - accuracy: 0.9136
354/375 [===========================>..] - ETA: 1s - loss: 0.2741 - accuracy: 0.9138
355/375 [===========================>..] - ETA: 0s - loss: 0.2737 - accuracy: 0.9140
356/375 [===========================>..] - ETA: 0s - loss: 0.2731 - accuracy: 0.9142
357/375 [===========================>..] - ETA: 0s - loss: 0.2726 - accuracy: 0.9143
358/375 [===========================>..] - ETA: 0s - loss: 0.2721 - accuracy: 0.9145
360/375 [===========================>..] - ETA: 0s - loss: 0.2711 - accuracy: 0.9148
361/375 [===========================>..] - ETA: 0s - loss: 0.2706 - accuracy: 0.9150
363/375 [============================>.] - ETA: 0s - loss: 0.2695 - accuracy: 0.9153
364/375 [============================>.] - ETA: 0s - loss: 0.2690 - accuracy: 0.9155
366/375 [============================>.] - ETA: 0s - loss: 0.2680 - accuracy: 0.9158
368/375 [============================>.] - ETA: 0s - loss: 0.2671 - accuracy: 0.9161
370/375 [============================>.] - ETA: 0s - loss: 0.2660 - accuracy: 0.9163
371/375 [============================>.] - ETA: 0s - loss: 0.2656 - accuracy: 0.9165
372/375 [============================>.] - ETA: 0s - loss: 0.2650 - accuracy: 0.9167
373/375 [============================>.] - ETA: 0s - loss: 0.2648 - accuracy: 0.9167
374/375 [============================>.] - ETA: 0s - loss: 0.2642 - accuracy: 0.9169
375/375 [==============================] - 20s 49ms/step - loss: 0.2638 - accuracy: 0.9170

375/375 [==============================] - 21s 53ms/step - loss: 0.2638 - accuracy: 0.9170 - val_loss: 0.0780 - val_accuracy: 0.9777
Epoch 2/5

  1/375 [..............................] - ETA: 24s - loss: 0.0540 - accuracy: 0.9688
  2/375 [..............................] - ETA: 23s - loss: 0.0471 - accuracy: 0.9766
  3/375 [..............................] - ETA: 24s - loss: 0.0537 - accuracy: 0.9792
  4/375 [..............................] - ETA: 24s - loss: 0.0523 - accuracy: 0.9805
  5/375 [..............................] - ETA: 23s - loss: 0.0457 - accuracy: 0.9844
  6/375 [..............................] - ETA: 22s - loss: 0.0460 - accuracy: 0.9844
  7/375 [..............................] - ETA: 23s - loss: 0.0464 - accuracy: 0.9844
  8/375 [..............................] - ETA: 22s - loss: 0.0469 - accuracy: 0.9844
  9/375 [..............................] - ETA: 22s - loss: 0.0585 - accuracy: 0.9809
 10/375 [..............................] - ETA: 22s - loss: 0.0607 - accuracy: 0.9789
 12/375 [..............................] - ETA: 21s - loss: 0.0753 - accuracy: 0.9740
 13/375 [>.............................] - ETA: 20s - loss: 0.0718 - accuracy: 0.9754
 14/375 [>.............................] - ETA: 21s - loss: 0.0700 - accuracy: 0.9766
 15/375 [>.............................] - ETA: 21s - loss: 0.0719 - accuracy: 0.9760
 16/375 [>.............................] - ETA: 21s - loss: 0.0695 - accuracy: 0.9771
 17/375 [>.............................] - ETA: 21s - loss: 0.0716 - accuracy: 0.9766
 18/375 [>.............................] - ETA: 21s - loss: 0.0699 - accuracy: 0.9774
 20/375 [>.............................] - ETA: 21s - loss: 0.0667 - accuracy: 0.9781
 22/375 [>.............................] - ETA: 21s - loss: 0.0680 - accuracy: 0.9769
 24/375 [>.............................] - ETA: 20s - loss: 0.0667 - accuracy: 0.9775
 25/375 [=>............................] - ETA: 20s - loss: 0.0659 - accuracy: 0.9778
 26/375 [=>............................] - ETA: 20s - loss: 0.0657 - accuracy: 0.9781
 27/375 [=>............................] - ETA: 20s - loss: 0.0652 - accuracy: 0.9780
 29/375 [=>............................] - ETA: 20s - loss: 0.0682 - accuracy: 0.9768
 31/375 [=>............................] - ETA: 20s - loss: 0.0757 - accuracy: 0.9753
 33/375 [=>............................] - ETA: 19s - loss: 0.0746 - accuracy: 0.9759
 35/375 [=>............................] - ETA: 19s - loss: 0.0733 - accuracy: 0.9766
 36/375 [=>............................] - ETA: 19s - loss: 0.0739 - accuracy: 0.9766
 37/375 [=>............................] - ETA: 19s - loss: 0.0776 - accuracy: 0.9753
 38/375 [==>...........................] - ETA: 19s - loss: 0.0791 - accuracy: 0.9751
 39/375 [==>...........................] - ETA: 19s - loss: 0.0785 - accuracy: 0.9752
 40/375 [==>...........................] - ETA: 19s - loss: 0.0782 - accuracy: 0.9756
 41/375 [==>...........................] - ETA: 19s - loss: 0.0788 - accuracy: 0.9756
 42/375 [==>...........................] - ETA: 19s - loss: 0.0781 - accuracy: 0.9760
 43/375 [==>...........................] - ETA: 19s - loss: 0.0787 - accuracy: 0.9757
 44/375 [==>...........................] - ETA: 19s - loss: 0.0800 - accuracy: 0.9753
 46/375 [==>...........................] - ETA: 19s - loss: 0.0792 - accuracy: 0.9754
 47/375 [==>...........................] - ETA: 19s - loss: 0.0788 - accuracy: 0.9756
 48/375 [==>...........................] - ETA: 19s - loss: 0.0792 - accuracy: 0.9756
 49/375 [==>...........................] - ETA: 19s - loss: 0.0791 - accuracy: 0.9753
 50/375 [===>..........................] - ETA: 19s - loss: 0.0781 - accuracy: 0.9758
 51/375 [===>..........................] - ETA: 19s - loss: 0.0773 - accuracy: 0.9759
 53/375 [===>..........................] - ETA: 18s - loss: 0.0758 - accuracy: 0.9763
 54/375 [===>..........................] - ETA: 18s - loss: 0.0751 - accuracy: 0.9766
 55/375 [===>..........................] - ETA: 18s - loss: 0.0751 - accuracy: 0.9763
 56/375 [===>..........................] - ETA: 18s - loss: 0.0742 - accuracy: 0.9766
 57/375 [===>..........................] - ETA: 18s - loss: 0.0732 - accuracy: 0.9770
 59/375 [===>..........................] - ETA: 18s - loss: 0.0721 - accuracy: 0.9774
 60/375 [===>..........................] - ETA: 18s - loss: 0.0712 - accuracy: 0.9777
 61/375 [===>..........................] - ETA: 18s - loss: 0.0703 - accuracy: 0.9781
 62/375 [===>..........................] - ETA: 18s - loss: 0.0718 - accuracy: 0.9779
 64/375 [====>.........................] - ETA: 18s - loss: 0.0709 - accuracy: 0.9784
 66/375 [====>.........................] - ETA: 18s - loss: 0.0722 - accuracy: 0.9777
 68/375 [====>.........................] - ETA: 17s - loss: 0.0716 - accuracy: 0.9778
 70/375 [====>.........................] - ETA: 17s - loss: 0.0717 - accuracy: 0.9779
 71/375 [====>.........................] - ETA: 17s - loss: 0.0716 - accuracy: 0.9780
 72/375 [====>.........................] - ETA: 17s - loss: 0.0715 - accuracy: 0.9781
 73/375 [====>.........................] - ETA: 17s - loss: 0.0710 - accuracy: 0.9783
 74/375 [====>.........................] - ETA: 17s - loss: 0.0716 - accuracy: 0.9779
 75/375 [=====>........................] - ETA: 17s - loss: 0.0716 - accuracy: 0.9779
 77/375 [=====>........................] - ETA: 17s - loss: 0.0707 - accuracy: 0.9781
 78/375 [=====>........................] - ETA: 17s - loss: 0.0702 - accuracy: 0.9783
 80/375 [=====>........................] - ETA: 17s - loss: 0.0711 - accuracy: 0.9780
 81/375 [=====>........................] - ETA: 17s - loss: 0.0709 - accuracy: 0.9780
 82/375 [=====>........................] - ETA: 17s - loss: 0.0711 - accuracy: 0.9781
 83/375 [=====>........................] - ETA: 17s - loss: 0.0708 - accuracy: 0.9781
 84/375 [=====>........................] - ETA: 17s - loss: 0.0713 - accuracy: 0.9781
 85/375 [=====>........................] - ETA: 17s - loss: 0.0708 - accuracy: 0.9782
 86/375 [=====>........................] - ETA: 16s - loss: 0.0707 - accuracy: 0.9783
 87/375 [=====>........................] - ETA: 16s - loss: 0.0712 - accuracy: 0.9780
 88/375 [======>.......................] - ETA: 16s - loss: 0.0721 - accuracy: 0.9776
 89/375 [======>.......................] - ETA: 16s - loss: 0.0724 - accuracy: 0.9775
 90/375 [======>.......................] - ETA: 16s - loss: 0.0724 - accuracy: 0.9773
 92/375 [======>.......................] - ETA: 16s - loss: 0.0722 - accuracy: 0.9774
 93/375 [======>.......................] - ETA: 16s - loss: 0.0721 - accuracy: 0.9776
 94/375 [======>.......................] - ETA: 16s - loss: 0.0715 - accuracy: 0.9778
 95/375 [======>.......................] - ETA: 16s - loss: 0.0718 - accuracy: 0.9779
 96/375 [======>.......................] - ETA: 16s - loss: 0.0714 - accuracy: 0.9779
 97/375 [======>.......................] - ETA: 16s - loss: 0.0714 - accuracy: 0.9781
 98/375 [======>.......................] - ETA: 16s - loss: 0.0712 - accuracy: 0.9780
 99/375 [======>.......................] - ETA: 16s - loss: 0.0711 - accuracy: 0.9781
101/375 [=======>......................] - ETA: 16s - loss: 0.0709 - accuracy: 0.9784
102/375 [=======>......................] - ETA: 15s - loss: 0.0706 - accuracy: 0.9785
103/375 [=======>......................] - ETA: 15s - loss: 0.0701 - accuracy: 0.9786
104/375 [=======>......................] - ETA: 15s - loss: 0.0706 - accuracy: 0.9784
106/375 [=======>......................] - ETA: 15s - loss: 0.0713 - accuracy: 0.9781
107/375 [=======>......................] - ETA: 15s - loss: 0.0710 - accuracy: 0.9782
109/375 [=======>......................] - ETA: 15s - loss: 0.0700 - accuracy: 0.9786
111/375 [=======>......................] - ETA: 15s - loss: 0.0702 - accuracy: 0.9786
113/375 [========>.....................] - ETA: 15s - loss: 0.0695 - accuracy: 0.9788
115/375 [========>.....................] - ETA: 15s - loss: 0.0688 - accuracy: 0.9791
116/375 [========>.....................] - ETA: 14s - loss: 0.0691 - accuracy: 0.9791
117/375 [========>.....................] - ETA: 14s - loss: 0.0695 - accuracy: 0.9790
118/375 [========>.....................] - ETA: 14s - loss: 0.0697 - accuracy: 0.9788
120/375 [========>.....................] - ETA: 14s - loss: 0.0701 - accuracy: 0.9788
121/375 [========>.....................] - ETA: 14s - loss: 0.0699 - accuracy: 0.9788
122/375 [========>.....................] - ETA: 14s - loss: 0.0705 - accuracy: 0.9786
123/375 [========>.....................] - ETA: 14s - loss: 0.0706 - accuracy: 0.9785
125/375 [=========>....................] - ETA: 14s - loss: 0.0705 - accuracy: 0.9785
127/375 [=========>....................] - ETA: 14s - loss: 0.0702 - accuracy: 0.9786
128/375 [=========>....................] - ETA: 14s - loss: 0.0699 - accuracy: 0.9786
130/375 [=========>....................] - ETA: 14s - loss: 0.0697 - accuracy: 0.9786
131/375 [=========>....................] - ETA: 14s - loss: 0.0696 - accuracy: 0.9786
132/375 [=========>....................] - ETA: 14s - loss: 0.0694 - accuracy: 0.9787
134/375 [=========>....................] - ETA: 13s - loss: 0.0697 - accuracy: 0.9785
135/375 [=========>....................] - ETA: 13s - loss: 0.0695 - accuracy: 0.9786
137/375 [=========>....................] - ETA: 13s - loss: 0.0698 - accuracy: 0.9785
139/375 [==========>...................] - ETA: 13s - loss: 0.0697 - accuracy: 0.9785
141/375 [==========>...................] - ETA: 13s - loss: 0.0692 - accuracy: 0.9786
143/375 [==========>...................] - ETA: 13s - loss: 0.0695 - accuracy: 0.9787
144/375 [==========>...................] - ETA: 13s - loss: 0.0695 - accuracy: 0.9787
146/375 [==========>...................] - ETA: 13s - loss: 0.0698 - accuracy: 0.9786
148/375 [==========>...................] - ETA: 12s - loss: 0.0704 - accuracy: 0.9785
150/375 [===========>..................] - ETA: 12s - loss: 0.0702 - accuracy: 0.9785
152/375 [===========>..................] - ETA: 12s - loss: 0.0701 - accuracy: 0.9785
154/375 [===========>..................] - ETA: 12s - loss: 0.0705 - accuracy: 0.9785
156/375 [===========>..................] - ETA: 12s - loss: 0.0705 - accuracy: 0.9786
158/375 [===========>..................] - ETA: 12s - loss: 0.0701 - accuracy: 0.9786
160/375 [===========>..................] - ETA: 12s - loss: 0.0697 - accuracy: 0.9787
161/375 [===========>..................] - ETA: 12s - loss: 0.0700 - accuracy: 0.9787
162/375 [===========>..................] - ETA: 11s - loss: 0.0701 - accuracy: 0.9787
164/375 [============>.................] - ETA: 11s - loss: 0.0700 - accuracy: 0.9787
165/375 [============>.................] - ETA: 11s - loss: 0.0698 - accuracy: 0.9787
166/375 [============>.................] - ETA: 11s - loss: 0.0696 - accuracy: 0.9788
167/375 [============>.................] - ETA: 11s - loss: 0.0696 - accuracy: 0.9788
169/375 [============>.................] - ETA: 11s - loss: 0.0694 - accuracy: 0.9788
170/375 [============>.................] - ETA: 11s - loss: 0.0697 - accuracy: 0.9788
171/375 [============>.................] - ETA: 11s - loss: 0.0696 - accuracy: 0.9788
172/375 [============>.................] - ETA: 11s - loss: 0.0696 - accuracy: 0.9789
174/375 [============>.................] - ETA: 11s - loss: 0.0693 - accuracy: 0.9790
176/375 [=============>................] - ETA: 11s - loss: 0.0688 - accuracy: 0.9792
178/375 [=============>................] - ETA: 11s - loss: 0.0688 - accuracy: 0.9792
180/375 [=============>................] - ETA: 10s - loss: 0.0684 - accuracy: 0.9793
182/375 [=============>................] - ETA: 10s - loss: 0.0682 - accuracy: 0.9794
183/375 [=============>................] - ETA: 10s - loss: 0.0681 - accuracy: 0.9794
184/375 [=============>................] - ETA: 10s - loss: 0.0680 - accuracy: 0.9794
185/375 [=============>................] - ETA: 10s - loss: 0.0678 - accuracy: 0.9795
186/375 [=============>................] - ETA: 10s - loss: 0.0676 - accuracy: 0.9795
187/375 [=============>................] - ETA: 10s - loss: 0.0675 - accuracy: 0.9796
188/375 [==============>...............] - ETA: 10s - loss: 0.0672 - accuracy: 0.9797
190/375 [==============>...............] - ETA: 10s - loss: 0.0671 - accuracy: 0.9797
191/375 [==============>...............] - ETA: 10s - loss: 0.0670 - accuracy: 0.9798
193/375 [==============>...............] - ETA: 10s - loss: 0.0666 - accuracy: 0.9799
194/375 [==============>...............] - ETA: 10s - loss: 0.0666 - accuracy: 0.9799
196/375 [==============>...............] - ETA: 9s - loss: 0.0670 - accuracy: 0.9797 
197/375 [==============>...............] - ETA: 9s - loss: 0.0668 - accuracy: 0.9797
198/375 [==============>...............] - ETA: 9s - loss: 0.0671 - accuracy: 0.9797
199/375 [==============>...............] - ETA: 9s - loss: 0.0669 - accuracy: 0.9798
200/375 [===============>..............] - ETA: 9s - loss: 0.0666 - accuracy: 0.9799
201/375 [===============>..............] - ETA: 9s - loss: 0.0664 - accuracy: 0.9799
202/375 [===============>..............] - ETA: 9s - loss: 0.0663 - accuracy: 0.9799
203/375 [===============>..............] - ETA: 9s - loss: 0.0662 - accuracy: 0.9799
205/375 [===============>..............] - ETA: 9s - loss: 0.0664 - accuracy: 0.9799
206/375 [===============>..............] - ETA: 9s - loss: 0.0665 - accuracy: 0.9799
207/375 [===============>..............] - ETA: 9s - loss: 0.0663 - accuracy: 0.9800
209/375 [===============>..............] - ETA: 9s - loss: 0.0662 - accuracy: 0.9800
210/375 [===============>..............] - ETA: 9s - loss: 0.0666 - accuracy: 0.9799
211/375 [===============>..............] - ETA: 9s - loss: 0.0667 - accuracy: 0.9799
213/375 [================>.............] - ETA: 9s - loss: 0.0668 - accuracy: 0.9798
214/375 [================>.............] - ETA: 9s - loss: 0.0669 - accuracy: 0.9798
216/375 [================>.............] - ETA: 8s - loss: 0.0672 - accuracy: 0.9798
217/375 [================>.............] - ETA: 8s - loss: 0.0674 - accuracy: 0.9798
219/375 [================>.............] - ETA: 8s - loss: 0.0672 - accuracy: 0.9798
221/375 [================>.............] - ETA: 8s - loss: 0.0673 - accuracy: 0.9797
223/375 [================>.............] - ETA: 8s - loss: 0.0669 - accuracy: 0.9798
225/375 [=================>............] - ETA: 8s - loss: 0.0666 - accuracy: 0.9799
226/375 [=================>............] - ETA: 8s - loss: 0.0665 - accuracy: 0.9800
228/375 [=================>............] - ETA: 8s - loss: 0.0665 - accuracy: 0.9800
230/375 [=================>............] - ETA: 8s - loss: 0.0668 - accuracy: 0.9799
232/375 [=================>............] - ETA: 7s - loss: 0.0670 - accuracy: 0.9798
233/375 [=================>............] - ETA: 7s - loss: 0.0674 - accuracy: 0.9797
235/375 [=================>............] - ETA: 7s - loss: 0.0672 - accuracy: 0.9798
237/375 [=================>............] - ETA: 7s - loss: 0.0669 - accuracy: 0.9798
238/375 [==================>...........] - ETA: 7s - loss: 0.0667 - accuracy: 0.9799
239/375 [==================>...........] - ETA: 7s - loss: 0.0668 - accuracy: 0.9799
240/375 [==================>...........] - ETA: 7s - loss: 0.0666 - accuracy: 0.9799
242/375 [==================>...........] - ETA: 7s - loss: 0.0669 - accuracy: 0.9798
244/375 [==================>...........] - ETA: 7s - loss: 0.0670 - accuracy: 0.9798
246/375 [==================>...........] - ETA: 7s - loss: 0.0673 - accuracy: 0.9797
247/375 [==================>...........] - ETA: 7s - loss: 0.0672 - accuracy: 0.9798
248/375 [==================>...........] - ETA: 7s - loss: 0.0670 - accuracy: 0.9798
250/375 [===================>..........] - ETA: 6s - loss: 0.0668 - accuracy: 0.9799
251/375 [===================>..........] - ETA: 6s - loss: 0.0666 - accuracy: 0.9799
253/375 [===================>..........] - ETA: 6s - loss: 0.0665 - accuracy: 0.9800
255/375 [===================>..........] - ETA: 6s - loss: 0.0663 - accuracy: 0.9800
256/375 [===================>..........] - ETA: 6s - loss: 0.0663 - accuracy: 0.9800
257/375 [===================>..........] - ETA: 6s - loss: 0.0665 - accuracy: 0.9800
259/375 [===================>..........] - ETA: 6s - loss: 0.0665 - accuracy: 0.9801
261/375 [===================>..........] - ETA: 6s - loss: 0.0665 - accuracy: 0.9800
263/375 [====================>.........] - ETA: 6s - loss: 0.0663 - accuracy: 0.9801
265/375 [====================>.........] - ETA: 6s - loss: 0.0663 - accuracy: 0.9800
267/375 [====================>.........] - ETA: 5s - loss: 0.0659 - accuracy: 0.9801
269/375 [====================>.........] - ETA: 5s - loss: 0.0657 - accuracy: 0.9802
271/375 [====================>.........] - ETA: 5s - loss: 0.0656 - accuracy: 0.9803
272/375 [====================>.........] - ETA: 5s - loss: 0.0655 - accuracy: 0.9803
273/375 [====================>.........] - ETA: 5s - loss: 0.0653 - accuracy: 0.9804
274/375 [====================>.........] - ETA: 5s - loss: 0.0651 - accuracy: 0.9804
275/375 [=====================>........] - ETA: 5s - loss: 0.0650 - accuracy: 0.9805
276/375 [=====================>........] - ETA: 5s - loss: 0.0649 - accuracy: 0.9805
277/375 [=====================>........] - ETA: 5s - loss: 0.0650 - accuracy: 0.9805
278/375 [=====================>........] - ETA: 5s - loss: 0.0649 - accuracy: 0.9805
280/375 [=====================>........] - ETA: 5s - loss: 0.0648 - accuracy: 0.9806
281/375 [=====================>........] - ETA: 5s - loss: 0.0648 - accuracy: 0.9806
282/375 [=====================>........] - ETA: 5s - loss: 0.0646 - accuracy: 0.9806
283/375 [=====================>........] - ETA: 5s - loss: 0.0647 - accuracy: 0.9806
285/375 [=====================>........] - ETA: 4s - loss: 0.0647 - accuracy: 0.9806
287/375 [=====================>........] - ETA: 4s - loss: 0.0648 - accuracy: 0.9805
288/375 [======================>.......] - ETA: 4s - loss: 0.0648 - accuracy: 0.9805
290/375 [======================>.......] - ETA: 4s - loss: 0.0644 - accuracy: 0.9806
291/375 [======================>.......] - ETA: 4s - loss: 0.0645 - accuracy: 0.9806
293/375 [======================>.......] - ETA: 4s - loss: 0.0644 - accuracy: 0.9806
295/375 [======================>.......] - ETA: 4s - loss: 0.0644 - accuracy: 0.9805
297/375 [======================>.......] - ETA: 4s - loss: 0.0646 - accuracy: 0.9805
299/375 [======================>.......] - ETA: 4s - loss: 0.0647 - accuracy: 0.9805
300/375 [=======================>......] - ETA: 4s - loss: 0.0648 - accuracy: 0.9805
302/375 [=======================>......] - ETA: 4s - loss: 0.0645 - accuracy: 0.9805
304/375 [=======================>......] - ETA: 3s - loss: 0.0647 - accuracy: 0.9804
306/375 [=======================>......] - ETA: 3s - loss: 0.0645 - accuracy: 0.9805
308/375 [=======================>......] - ETA: 3s - loss: 0.0642 - accuracy: 0.9805
309/375 [=======================>......] - ETA: 3s - loss: 0.0641 - accuracy: 0.9805
310/375 [=======================>......] - ETA: 3s - loss: 0.0640 - accuracy: 0.9806
312/375 [=======================>......] - ETA: 3s - loss: 0.0640 - accuracy: 0.9806
313/375 [========================>.....] - ETA: 3s - loss: 0.0641 - accuracy: 0.9806
314/375 [========================>.....] - ETA: 3s - loss: 0.0643 - accuracy: 0.9805
315/375 [========================>.....] - ETA: 3s - loss: 0.0642 - accuracy: 0.9806
317/375 [========================>.....] - ETA: 3s - loss: 0.0640 - accuracy: 0.9806
319/375 [========================>.....] - ETA: 3s - loss: 0.0643 - accuracy: 0.9806
320/375 [========================>.....] - ETA: 3s - loss: 0.0644 - accuracy: 0.9806
321/375 [========================>.....] - ETA: 2s - loss: 0.0642 - accuracy: 0.9807
322/375 [========================>.....] - ETA: 2s - loss: 0.0640 - accuracy: 0.9807
323/375 [========================>.....] - ETA: 2s - loss: 0.0639 - accuracy: 0.9807
324/375 [========================>.....] - ETA: 2s - loss: 0.0638 - accuracy: 0.9808
326/375 [=========================>....] - ETA: 2s - loss: 0.0635 - accuracy: 0.9809
328/375 [=========================>....] - ETA: 2s - loss: 0.0634 - accuracy: 0.9808
330/375 [=========================>....] - ETA: 2s - loss: 0.0633 - accuracy: 0.9809
332/375 [=========================>....] - ETA: 2s - loss: 0.0632 - accuracy: 0.9809
334/375 [=========================>....] - ETA: 2s - loss: 0.0630 - accuracy: 0.9809
336/375 [=========================>....] - ETA: 2s - loss: 0.0628 - accuracy: 0.9810
337/375 [=========================>....] - ETA: 2s - loss: 0.0626 - accuracy: 0.9811
339/375 [==========================>...] - ETA: 1s - loss: 0.0624 - accuracy: 0.9811
340/375 [==========================>...] - ETA: 1s - loss: 0.0623 - accuracy: 0.9812
342/375 [==========================>...] - ETA: 1s - loss: 0.0624 - accuracy: 0.9812
343/375 [==========================>...] - ETA: 1s - loss: 0.0628 - accuracy: 0.9811
344/375 [==========================>...] - ETA: 1s - loss: 0.0629 - accuracy: 0.9810
345/375 [==========================>...] - ETA: 1s - loss: 0.0631 - accuracy: 0.9810
346/375 [==========================>...] - ETA: 1s - loss: 0.0630 - accuracy: 0.9809
347/375 [==========================>...] - ETA: 1s - loss: 0.0629 - accuracy: 0.9810
349/375 [==========================>...] - ETA: 1s - loss: 0.0629 - accuracy: 0.9810
350/375 [===========================>..] - ETA: 1s - loss: 0.0630 - accuracy: 0.9810
351/375 [===========================>..] - ETA: 1s - loss: 0.0629 - accuracy: 0.9811
353/375 [===========================>..] - ETA: 1s - loss: 0.0626 - accuracy: 0.9811
354/375 [===========================>..] - ETA: 1s - loss: 0.0626 - accuracy: 0.9811
355/375 [===========================>..] - ETA: 1s - loss: 0.0625 - accuracy: 0.9811
356/375 [===========================>..] - ETA: 1s - loss: 0.0626 - accuracy: 0.9811
357/375 [===========================>..] - ETA: 0s - loss: 0.0625 - accuracy: 0.9812
358/375 [===========================>..] - ETA: 0s - loss: 0.0626 - accuracy: 0.9812
359/375 [===========================>..] - ETA: 0s - loss: 0.0624 - accuracy: 0.9812
361/375 [===========================>..] - ETA: 0s - loss: 0.0626 - accuracy: 0.9812
362/375 [===========================>..] - ETA: 0s - loss: 0.0627 - accuracy: 0.9811
364/375 [============================>.] - ETA: 0s - loss: 0.0624 - accuracy: 0.9812
366/375 [============================>.] - ETA: 0s - loss: 0.0624 - accuracy: 0.9812
368/375 [============================>.] - ETA: 0s - loss: 0.0627 - accuracy: 0.9811
370/375 [============================>.] - ETA: 0s - loss: 0.0628 - accuracy: 0.9810
372/375 [============================>.] - ETA: 0s - loss: 0.0628 - accuracy: 0.9810
374/375 [============================>.] - ETA: 0s - loss: 0.0629 - accuracy: 0.9811
375/375 [==============================] - 20s 54ms/step - loss: 0.0628 - accuracy: 0.9811

375/375 [==============================] - 22s 58ms/step - loss: 0.0628 - accuracy: 0.9811 - val_loss: 0.0579 - val_accuracy: 0.9826
Epoch 3/5

  1/375 [..............................] - ETA: 11s - loss: 0.0878 - accuracy: 0.9688
  3/375 [..............................] - ETA: 11s - loss: 0.0552 - accuracy: 0.9818
  5/375 [..............................] - ETA: 11s - loss: 0.0473 - accuracy: 0.9844
  7/375 [..............................] - ETA: 10s - loss: 0.0449 - accuracy: 0.9866
  9/375 [..............................] - ETA: 11s - loss: 0.0409 - accuracy: 0.9887
 11/375 [..............................] - ETA: 10s - loss: 0.0357 - accuracy: 0.9908
 13/375 [>.............................] - ETA: 11s - loss: 0.0384 - accuracy: 0.9904
 15/375 [>.............................] - ETA: 11s - loss: 0.0407 - accuracy: 0.9891
 17/375 [>.............................] - ETA: 11s - loss: 0.0436 - accuracy: 0.9881
 19/375 [>.............................] - ETA: 10s - loss: 0.0408 - accuracy: 0.9889
 21/375 [>.............................] - ETA: 10s - loss: 0.0416 - accuracy: 0.9885
 23/375 [>.............................] - ETA: 10s - loss: 0.0396 - accuracy: 0.9891
 25/375 [=>............................] - ETA: 10s - loss: 0.0380 - accuracy: 0.9897
 27/375 [=>............................] - ETA: 10s - loss: 0.0387 - accuracy: 0.9890
 29/375 [=>............................] - ETA: 10s - loss: 0.0420 - accuracy: 0.9879
 30/375 [=>............................] - ETA: 11s - loss: 0.0412 - accuracy: 0.9883
 32/375 [=>............................] - ETA: 11s - loss: 0.0447 - accuracy: 0.9875
 34/375 [=>............................] - ETA: 11s - loss: 0.0442 - accuracy: 0.9878
 36/375 [=>............................] - ETA: 10s - loss: 0.0423 - accuracy: 0.9883
 38/375 [==>...........................] - ETA: 10s - loss: 0.0419 - accuracy: 0.9885
 40/375 [==>...........................] - ETA: 10s - loss: 0.0418 - accuracy: 0.9887
 42/375 [==>...........................] - ETA: 10s - loss: 0.0421 - accuracy: 0.9885
 44/375 [==>...........................] - ETA: 10s - loss: 0.0433 - accuracy: 0.9879
 46/375 [==>...........................] - ETA: 10s - loss: 0.0429 - accuracy: 0.9881
 48/375 [==>...........................] - ETA: 10s - loss: 0.0416 - accuracy: 0.9884
 50/375 [===>..........................] - ETA: 10s - loss: 0.0423 - accuracy: 0.9881
 52/375 [===>..........................] - ETA: 10s - loss: 0.0415 - accuracy: 0.9884
 54/375 [===>..........................] - ETA: 10s - loss: 0.0412 - accuracy: 0.9884
 56/375 [===>..........................] - ETA: 10s - loss: 0.0413 - accuracy: 0.9883
 58/375 [===>..........................] - ETA: 10s - loss: 0.0408 - accuracy: 0.9883
 60/375 [===>..........................] - ETA: 10s - loss: 0.0404 - accuracy: 0.9882
 62/375 [===>..........................] - ETA: 10s - loss: 0.0403 - accuracy: 0.9880
 64/375 [====>.........................] - ETA: 10s - loss: 0.0411 - accuracy: 0.9877
 66/375 [====>.........................] - ETA: 10s - loss: 0.0413 - accuracy: 0.9873
 68/375 [====>.........................] - ETA: 10s - loss: 0.0423 - accuracy: 0.9868
 70/375 [====>.........................] - ETA: 9s - loss: 0.0431 - accuracy: 0.9866 
 72/375 [====>.........................] - ETA: 9s - loss: 0.0434 - accuracy: 0.9865
 74/375 [====>.........................] - ETA: 9s - loss: 0.0431 - accuracy: 0.9866
 76/375 [=====>........................] - ETA: 9s - loss: 0.0431 - accuracy: 0.9866
 78/375 [=====>........................] - ETA: 9s - loss: 0.0431 - accuracy: 0.9865
 79/375 [=====>........................] - ETA: 9s - loss: 0.0432 - accuracy: 0.9864
 80/375 [=====>........................] - ETA: 10s - loss: 0.0432 - accuracy: 0.9864
 81/375 [=====>........................] - ETA: 10s - loss: 0.0432 - accuracy: 0.9865
 82/375 [=====>........................] - ETA: 10s - loss: 0.0429 - accuracy: 0.9866
 83/375 [=====>........................] - ETA: 10s - loss: 0.0428 - accuracy: 0.9864
 84/375 [=====>........................] - ETA: 10s - loss: 0.0428 - accuracy: 0.9863
 85/375 [=====>........................] - ETA: 10s - loss: 0.0428 - accuracy: 0.9864
 86/375 [=====>........................] - ETA: 10s - loss: 0.0430 - accuracy: 0.9863
 87/375 [=====>........................] - ETA: 10s - loss: 0.0430 - accuracy: 0.9861
 88/375 [======>.......................] - ETA: 10s - loss: 0.0430 - accuracy: 0.9860
 89/375 [======>.......................] - ETA: 10s - loss: 0.0429 - accuracy: 0.9860
 90/375 [======>.......................] - ETA: 11s - loss: 0.0430 - accuracy: 0.9859
 91/375 [======>.......................] - ETA: 11s - loss: 0.0429 - accuracy: 0.9860
 92/375 [======>.......................] - ETA: 11s - loss: 0.0435 - accuracy: 0.9858
 94/375 [======>.......................] - ETA: 11s - loss: 0.0430 - accuracy: 0.9860
 96/375 [======>.......................] - ETA: 11s - loss: 0.0425 - accuracy: 0.9862
 97/375 [======>.......................] - ETA: 11s - loss: 0.0422 - accuracy: 0.9863
 98/375 [======>.......................] - ETA: 11s - loss: 0.0421 - accuracy: 0.9864
100/375 [=======>......................] - ETA: 11s - loss: 0.0433 - accuracy: 0.9858
102/375 [=======>......................] - ETA: 11s - loss: 0.0433 - accuracy: 0.9858
104/375 [=======>......................] - ETA: 10s - loss: 0.0429 - accuracy: 0.9859
105/375 [=======>......................] - ETA: 10s - loss: 0.0426 - accuracy: 0.9860
107/375 [=======>......................] - ETA: 10s - loss: 0.0420 - accuracy: 0.9863
108/375 [=======>......................] - ETA: 10s - loss: 0.0419 - accuracy: 0.9863
109/375 [=======>......................] - ETA: 10s - loss: 0.0417 - accuracy: 0.9864
111/375 [=======>......................] - ETA: 10s - loss: 0.0416 - accuracy: 0.9865
112/375 [=======>......................] - ETA: 10s - loss: 0.0416 - accuracy: 0.9865
113/375 [========>.....................] - ETA: 10s - loss: 0.0416 - accuracy: 0.9864
115/375 [========>.....................] - ETA: 10s - loss: 0.0413 - accuracy: 0.9865
117/375 [========>.....................] - ETA: 10s - loss: 0.0411 - accuracy: 0.9866
119/375 [========>.....................] - ETA: 10s - loss: 0.0415 - accuracy: 0.9865
120/375 [========>.....................] - ETA: 10s - loss: 0.0418 - accuracy: 0.9863
121/375 [========>.....................] - ETA: 10s - loss: 0.0419 - accuracy: 0.9863
122/375 [========>.....................] - ETA: 10s - loss: 0.0421 - accuracy: 0.9862
123/375 [========>.....................] - ETA: 10s - loss: 0.0422 - accuracy: 0.9862
124/375 [========>.....................] - ETA: 10s - loss: 0.0423 - accuracy: 0.9861
125/375 [=========>....................] - ETA: 10s - loss: 0.0421 - accuracy: 0.9862
126/375 [=========>....................] - ETA: 10s - loss: 0.0419 - accuracy: 0.9864
128/375 [=========>....................] - ETA: 10s - loss: 0.0416 - accuracy: 0.9865
129/375 [=========>....................] - ETA: 10s - loss: 0.0418 - accuracy: 0.9864
130/375 [=========>....................] - ETA: 10s - loss: 0.0417 - accuracy: 0.9864
132/375 [=========>....................] - ETA: 10s - loss: 0.0415 - accuracy: 0.9864
133/375 [=========>....................] - ETA: 10s - loss: 0.0413 - accuracy: 0.9865
134/375 [=========>....................] - ETA: 10s - loss: 0.0411 - accuracy: 0.9866
135/375 [=========>....................] - ETA: 10s - loss: 0.0408 - accuracy: 0.9867
136/375 [=========>....................] - ETA: 10s - loss: 0.0406 - accuracy: 0.9867
138/375 [==========>...................] - ETA: 10s - loss: 0.0407 - accuracy: 0.9866
140/375 [==========>...................] - ETA: 10s - loss: 0.0403 - accuracy: 0.9868
141/375 [==========>...................] - ETA: 10s - loss: 0.0402 - accuracy: 0.9868
142/375 [==========>...................] - ETA: 10s - loss: 0.0403 - accuracy: 0.9868
144/375 [==========>...................] - ETA: 10s - loss: 0.0411 - accuracy: 0.9867
146/375 [==========>...................] - ETA: 10s - loss: 0.0413 - accuracy: 0.9865
147/375 [==========>...................] - ETA: 10s - loss: 0.0417 - accuracy: 0.9864
148/375 [==========>...................] - ETA: 10s - loss: 0.0417 - accuracy: 0.9863
149/375 [==========>...................] - ETA: 10s - loss: 0.0423 - accuracy: 0.9862
150/375 [===========>..................] - ETA: 10s - loss: 0.0425 - accuracy: 0.9861
151/375 [===========>..................] - ETA: 10s - loss: 0.0425 - accuracy: 0.9861
152/375 [===========>..................] - ETA: 10s - loss: 0.0425 - accuracy: 0.9861
154/375 [===========>..................] - ETA: 10s - loss: 0.0427 - accuracy: 0.9861
156/375 [===========>..................] - ETA: 10s - loss: 0.0424 - accuracy: 0.9862
157/375 [===========>..................] - ETA: 9s - loss: 0.0422 - accuracy: 0.9863 
158/375 [===========>..................] - ETA: 9s - loss: 0.0421 - accuracy: 0.9863
159/375 [===========>..................] - ETA: 9s - loss: 0.0421 - accuracy: 0.9863
161/375 [===========>..................] - ETA: 9s - loss: 0.0422 - accuracy: 0.9863
163/375 [============>.................] - ETA: 9s - loss: 0.0422 - accuracy: 0.9863
164/375 [============>.................] - ETA: 9s - loss: 0.0424 - accuracy: 0.9862
165/375 [============>.................] - ETA: 9s - loss: 0.0427 - accuracy: 0.9861
166/375 [============>.................] - ETA: 9s - loss: 0.0427 - accuracy: 0.9861
167/375 [============>.................] - ETA: 9s - loss: 0.0425 - accuracy: 0.9862
168/375 [============>.................] - ETA: 9s - loss: 0.0424 - accuracy: 0.9862
169/375 [============>.................] - ETA: 9s - loss: 0.0424 - accuracy: 0.9863
171/375 [============>.................] - ETA: 9s - loss: 0.0423 - accuracy: 0.9862
173/375 [============>.................] - ETA: 9s - loss: 0.0424 - accuracy: 0.9862
175/375 [=============>................] - ETA: 9s - loss: 0.0422 - accuracy: 0.9863
176/375 [=============>................] - ETA: 9s - loss: 0.0420 - accuracy: 0.9864
178/375 [=============>................] - ETA: 9s - loss: 0.0418 - accuracy: 0.9865
180/375 [=============>................] - ETA: 9s - loss: 0.0414 - accuracy: 0.9866
181/375 [=============>................] - ETA: 9s - loss: 0.0414 - accuracy: 0.9867
182/375 [=============>................] - ETA: 9s - loss: 0.0413 - accuracy: 0.9867
183/375 [=============>................] - ETA: 9s - loss: 0.0412 - accuracy: 0.9867
184/375 [=============>................] - ETA: 9s - loss: 0.0414 - accuracy: 0.9866
185/375 [=============>................] - ETA: 8s - loss: 0.0415 - accuracy: 0.9865
186/375 [=============>................] - ETA: 8s - loss: 0.0415 - accuracy: 0.9865
187/375 [=============>................] - ETA: 8s - loss: 0.0418 - accuracy: 0.9864
188/375 [==============>...............] - ETA: 8s - loss: 0.0421 - accuracy: 0.9864
189/375 [==============>...............] - ETA: 8s - loss: 0.0419 - accuracy: 0.9864
191/375 [==============>...............] - ETA: 8s - loss: 0.0420 - accuracy: 0.9864
192/375 [==============>...............] - ETA: 8s - loss: 0.0420 - accuracy: 0.9864
193/375 [==============>...............] - ETA: 8s - loss: 0.0422 - accuracy: 0.9862
194/375 [==============>...............] - ETA: 8s - loss: 0.0421 - accuracy: 0.9862
195/375 [==============>...............] - ETA: 8s - loss: 0.0421 - accuracy: 0.9862
197/375 [==============>...............] - ETA: 8s - loss: 0.0418 - accuracy: 0.9864
199/375 [==============>...............] - ETA: 8s - loss: 0.0418 - accuracy: 0.9863
200/375 [===============>..............] - ETA: 8s - loss: 0.0419 - accuracy: 0.9863
201/375 [===============>..............] - ETA: 8s - loss: 0.0419 - accuracy: 0.9863
203/375 [===============>..............] - ETA: 8s - loss: 0.0416 - accuracy: 0.9865
205/375 [===============>..............] - ETA: 8s - loss: 0.0415 - accuracy: 0.9865
206/375 [===============>..............] - ETA: 8s - loss: 0.0413 - accuracy: 0.9866
207/375 [===============>..............] - ETA: 8s - loss: 0.0412 - accuracy: 0.9866
209/375 [===============>..............] - ETA: 8s - loss: 0.0410 - accuracy: 0.9866
210/375 [===============>..............] - ETA: 8s - loss: 0.0409 - accuracy: 0.9867
211/375 [===============>..............] - ETA: 7s - loss: 0.0409 - accuracy: 0.9866
213/375 [================>.............] - ETA: 7s - loss: 0.0407 - accuracy: 0.9867
214/375 [================>.............] - ETA: 7s - loss: 0.0407 - accuracy: 0.9867
215/375 [================>.............] - ETA: 7s - loss: 0.0409 - accuracy: 0.9867
217/375 [================>.............] - ETA: 7s - loss: 0.0410 - accuracy: 0.9866
219/375 [================>.............] - ETA: 7s - loss: 0.0409 - accuracy: 0.9866
221/375 [================>.............] - ETA: 7s - loss: 0.0406 - accuracy: 0.9867
222/375 [================>.............] - ETA: 7s - loss: 0.0405 - accuracy: 0.9867
224/375 [================>.............] - ETA: 7s - loss: 0.0404 - accuracy: 0.9867
226/375 [=================>............] - ETA: 7s - loss: 0.0403 - accuracy: 0.9867
227/375 [=================>............] - ETA: 7s - loss: 0.0405 - accuracy: 0.9866
228/375 [=================>............] - ETA: 7s - loss: 0.0406 - accuracy: 0.9866
230/375 [=================>............] - ETA: 7s - loss: 0.0410 - accuracy: 0.9865
232/375 [=================>............] - ETA: 7s - loss: 0.0412 - accuracy: 0.9865
234/375 [=================>............] - ETA: 6s - loss: 0.0411 - accuracy: 0.9865
236/375 [=================>............] - ETA: 6s - loss: 0.0409 - accuracy: 0.9866
237/375 [=================>............] - ETA: 6s - loss: 0.0412 - accuracy: 0.9866
238/375 [==================>...........] - ETA: 6s - loss: 0.0412 - accuracy: 0.9866
239/375 [==================>...........] - ETA: 6s - loss: 0.0414 - accuracy: 0.9865
240/375 [==================>...........] - ETA: 6s - loss: 0.0413 - accuracy: 0.9866
242/375 [==================>...........] - ETA: 6s - loss: 0.0413 - accuracy: 0.9866
244/375 [==================>...........] - ETA: 6s - loss: 0.0414 - accuracy: 0.9865
245/375 [==================>...........] - ETA: 6s - loss: 0.0414 - accuracy: 0.9865
246/375 [==================>...........] - ETA: 6s - loss: 0.0413 - accuracy: 0.9865
247/375 [==================>...........] - ETA: 6s - loss: 0.0412 - accuracy: 0.9866
248/375 [==================>...........] - ETA: 6s - loss: 0.0411 - accuracy: 0.9866
249/375 [==================>...........] - ETA: 6s - loss: 0.0414 - accuracy: 0.9865
251/375 [===================>..........] - ETA: 6s - loss: 0.0414 - accuracy: 0.9865
252/375 [===================>..........] - ETA: 6s - loss: 0.0415 - accuracy: 0.9865
254/375 [===================>..........] - ETA: 6s - loss: 0.0413 - accuracy: 0.9865
256/375 [===================>..........] - ETA: 5s - loss: 0.0413 - accuracy: 0.9865
258/375 [===================>..........] - ETA: 5s - loss: 0.0414 - accuracy: 0.9865
260/375 [===================>..........] - ETA: 5s - loss: 0.0414 - accuracy: 0.9865
261/375 [===================>..........] - ETA: 5s - loss: 0.0414 - accuracy: 0.9865
262/375 [===================>..........] - ETA: 5s - loss: 0.0414 - accuracy: 0.9865
263/375 [====================>.........] - ETA: 5s - loss: 0.0413 - accuracy: 0.9865
264/375 [====================>.........] - ETA: 5s - loss: 0.0413 - accuracy: 0.9865
265/375 [====================>.........] - ETA: 5s - loss: 0.0412 - accuracy: 0.9865
266/375 [====================>.........] - ETA: 5s - loss: 0.0415 - accuracy: 0.9864
267/375 [====================>.........] - ETA: 5s - loss: 0.0417 - accuracy: 0.9863
268/375 [====================>.........] - ETA: 5s - loss: 0.0418 - accuracy: 0.9863
269/375 [====================>.........] - ETA: 5s - loss: 0.0417 - accuracy: 0.9863
271/375 [====================>.........] - ETA: 5s - loss: 0.0415 - accuracy: 0.9864
272/375 [====================>.........] - ETA: 5s - loss: 0.0416 - accuracy: 0.9864
273/375 [====================>.........] - ETA: 5s - loss: 0.0414 - accuracy: 0.9864
274/375 [====================>.........] - ETA: 5s - loss: 0.0414 - accuracy: 0.9864
275/375 [=====================>........] - ETA: 5s - loss: 0.0415 - accuracy: 0.9864
276/375 [=====================>........] - ETA: 5s - loss: 0.0414 - accuracy: 0.9865
277/375 [=====================>........] - ETA: 4s - loss: 0.0412 - accuracy: 0.9865
278/375 [=====================>........] - ETA: 4s - loss: 0.0411 - accuracy: 0.9866
279/375 [=====================>........] - ETA: 4s - loss: 0.0410 - accuracy: 0.9866
280/375 [=====================>........] - ETA: 4s - loss: 0.0412 - accuracy: 0.9866
282/375 [=====================>........] - ETA: 4s - loss: 0.0413 - accuracy: 0.9865
284/375 [=====================>........] - ETA: 4s - loss: 0.0416 - accuracy: 0.9865
286/375 [=====================>........] - ETA: 4s - loss: 0.0416 - accuracy: 0.9865
288/375 [======================>.......] - ETA: 4s - loss: 0.0414 - accuracy: 0.9865
290/375 [======================>.......] - ETA: 4s - loss: 0.0412 - accuracy: 0.9866
291/375 [======================>.......] - ETA: 4s - loss: 0.0413 - accuracy: 0.9866
292/375 [======================>.......] - ETA: 4s - loss: 0.0413 - accuracy: 0.9866
293/375 [======================>.......] - ETA: 4s - loss: 0.0412 - accuracy: 0.9866
294/375 [======================>.......] - ETA: 4s - loss: 0.0412 - accuracy: 0.9866
295/375 [======================>.......] - ETA: 4s - loss: 0.0415 - accuracy: 0.9866
296/375 [======================>.......] - ETA: 4s - loss: 0.0414 - accuracy: 0.9866
297/375 [======================>.......] - ETA: 3s - loss: 0.0416 - accuracy: 0.9866
298/375 [======================>.......] - ETA: 3s - loss: 0.0415 - accuracy: 0.9866
299/375 [======================>.......] - ETA: 3s - loss: 0.0415 - accuracy: 0.9866
300/375 [=======================>......] - ETA: 3s - loss: 0.0415 - accuracy: 0.9866
301/375 [=======================>......] - ETA: 3s - loss: 0.0414 - accuracy: 0.9867
302/375 [=======================>......] - ETA: 3s - loss: 0.0414 - accuracy: 0.9867
303/375 [=======================>......] - ETA: 3s - loss: 0.0413 - accuracy: 0.9867
304/375 [=======================>......] - ETA: 3s - loss: 0.0414 - accuracy: 0.9867
305/375 [=======================>......] - ETA: 3s - loss: 0.0414 - accuracy: 0.9867
306/375 [=======================>......] - ETA: 3s - loss: 0.0413 - accuracy: 0.9867
308/375 [=======================>......] - ETA: 3s - loss: 0.0412 - accuracy: 0.9867
310/375 [=======================>......] - ETA: 3s - loss: 0.0413 - accuracy: 0.9866
311/375 [=======================>......] - ETA: 3s - loss: 0.0412 - accuracy: 0.9866
312/375 [=======================>......] - ETA: 3s - loss: 0.0412 - accuracy: 0.9867
314/375 [========================>.....] - ETA: 3s - loss: 0.0412 - accuracy: 0.9866
315/375 [========================>.....] - ETA: 3s - loss: 0.0411 - accuracy: 0.9867
317/375 [========================>.....] - ETA: 3s - loss: 0.0410 - accuracy: 0.9867
318/375 [========================>.....] - ETA: 2s - loss: 0.0410 - accuracy: 0.9867
319/375 [========================>.....] - ETA: 2s - loss: 0.0411 - accuracy: 0.9866
320/375 [========================>.....] - ETA: 2s - loss: 0.0409 - accuracy: 0.9866
322/375 [========================>.....] - ETA: 2s - loss: 0.0410 - accuracy: 0.9867
323/375 [========================>.....] - ETA: 2s - loss: 0.0408 - accuracy: 0.9867
324/375 [========================>.....] - ETA: 2s - loss: 0.0410 - accuracy: 0.9867
325/375 [=========================>....] - ETA: 2s - loss: 0.0410 - accuracy: 0.9867
326/375 [=========================>....] - ETA: 2s - loss: 0.0409 - accuracy: 0.9867
327/375 [=========================>....] - ETA: 2s - loss: 0.0411 - accuracy: 0.9866
328/375 [=========================>....] - ETA: 2s - loss: 0.0412 - accuracy: 0.9865
329/375 [=========================>....] - ETA: 2s - loss: 0.0411 - accuracy: 0.9866
330/375 [=========================>....] - ETA: 2s - loss: 0.0411 - accuracy: 0.9866
331/375 [=========================>....] - ETA: 2s - loss: 0.0410 - accuracy: 0.9866
332/375 [=========================>....] - ETA: 2s - loss: 0.0412 - accuracy: 0.9866
333/375 [=========================>....] - ETA: 2s - loss: 0.0411 - accuracy: 0.9866
334/375 [=========================>....] - ETA: 2s - loss: 0.0410 - accuracy: 0.9866
335/375 [=========================>....] - ETA: 2s - loss: 0.0410 - accuracy: 0.9866
336/375 [=========================>....] - ETA: 2s - loss: 0.0410 - accuracy: 0.9866
337/375 [=========================>....] - ETA: 2s - loss: 0.0410 - accuracy: 0.9866
338/375 [==========================>...] - ETA: 1s - loss: 0.0409 - accuracy: 0.9866
339/375 [==========================>...] - ETA: 1s - loss: 0.0409 - accuracy: 0.9866
341/375 [==========================>...] - ETA: 1s - loss: 0.0410 - accuracy: 0.9866
342/375 [==========================>...] - ETA: 1s - loss: 0.0410 - accuracy: 0.9866
343/375 [==========================>...] - ETA: 1s - loss: 0.0410 - accuracy: 0.9866
344/375 [==========================>...] - ETA: 1s - loss: 0.0409 - accuracy: 0.9866
345/375 [==========================>...] - ETA: 1s - loss: 0.0408 - accuracy: 0.9867
346/375 [==========================>...] - ETA: 1s - loss: 0.0409 - accuracy: 0.9867
347/375 [==========================>...] - ETA: 1s - loss: 0.0409 - accuracy: 0.9867
348/375 [==========================>...] - ETA: 1s - loss: 0.0410 - accuracy: 0.9866
349/375 [==========================>...] - ETA: 1s - loss: 0.0409 - accuracy: 0.9867
350/375 [===========================>..] - ETA: 1s - loss: 0.0408 - accuracy: 0.9867
351/375 [===========================>..] - ETA: 1s - loss: 0.0408 - accuracy: 0.9867
352/375 [===========================>..] - ETA: 1s - loss: 0.0408 - accuracy: 0.9867
353/375 [===========================>..] - ETA: 1s - loss: 0.0408 - accuracy: 0.9867
354/375 [===========================>..] - ETA: 1s - loss: 0.0407 - accuracy: 0.9868
355/375 [===========================>..] - ETA: 1s - loss: 0.0406 - accuracy: 0.9868
356/375 [===========================>..] - ETA: 1s - loss: 0.0406 - accuracy: 0.9868
357/375 [===========================>..] - ETA: 0s - loss: 0.0406 - accuracy: 0.9868
358/375 [===========================>..] - ETA: 0s - loss: 0.0405 - accuracy: 0.9868
359/375 [===========================>..] - ETA: 0s - loss: 0.0406 - accuracy: 0.9868
360/375 [===========================>..] - ETA: 0s - loss: 0.0405 - accuracy: 0.9868
361/375 [===========================>..] - ETA: 0s - loss: 0.0405 - accuracy: 0.9868
362/375 [===========================>..] - ETA: 0s - loss: 0.0404 - accuracy: 0.9868
363/375 [============================>.] - ETA: 0s - loss: 0.0404 - accuracy: 0.9868
364/375 [============================>.] - ETA: 0s - loss: 0.0406 - accuracy: 0.9867
365/375 [============================>.] - ETA: 0s - loss: 0.0405 - accuracy: 0.9867
366/375 [============================>.] - ETA: 0s - loss: 0.0407 - accuracy: 0.9867
368/375 [============================>.] - ETA: 0s - loss: 0.0410 - accuracy: 0.9866
369/375 [============================>.] - ETA: 0s - loss: 0.0409 - accuracy: 0.9866
370/375 [============================>.] - ETA: 0s - loss: 0.0409 - accuracy: 0.9867
371/375 [============================>.] - ETA: 0s - loss: 0.0409 - accuracy: 0.9866
372/375 [============================>.] - ETA: 0s - loss: 0.0409 - accuracy: 0.9866
373/375 [============================>.] - ETA: 0s - loss: 0.0409 - accuracy: 0.9866
375/375 [==============================] - 20s 54ms/step - loss: 0.0410 - accuracy: 0.9865

375/375 [==============================] - 22s 58ms/step - loss: 0.0410 - accuracy: 0.9865 - val_loss: 0.0438 - val_accuracy: 0.9865
Epoch 4/5

  1/375 [..............................] - ETA: 15s - loss: 0.0136 - accuracy: 1.0000
  3/375 [..............................] - ETA: 12s - loss: 0.0193 - accuracy: 0.9948
  5/375 [..............................] - ETA: 12s - loss: 0.0188 - accuracy: 0.9937
  7/375 [..............................] - ETA: 12s - loss: 0.0198 - accuracy: 0.9922
  9/375 [..............................] - ETA: 12s - loss: 0.0260 - accuracy: 0.9896
 11/375 [..............................] - ETA: 12s - loss: 0.0300 - accuracy: 0.9886
 13/375 [>.............................] - ETA: 12s - loss: 0.0324 - accuracy: 0.9868
 15/375 [>.............................] - ETA: 12s - loss: 0.0307 - accuracy: 0.9880
 17/375 [>.............................] - ETA: 12s - loss: 0.0293 - accuracy: 0.9890
 19/375 [>.............................] - ETA: 11s - loss: 0.0288 - accuracy: 0.9893
 21/375 [>.............................] - ETA: 11s - loss: 0.0282 - accuracy: 0.9892
 23/375 [>.............................] - ETA: 11s - loss: 0.0268 - accuracy: 0.9901
 25/375 [=>............................] - ETA: 12s - loss: 0.0286 - accuracy: 0.9900
 27/375 [=>............................] - ETA: 12s - loss: 0.0314 - accuracy: 0.9893
 29/375 [=>............................] - ETA: 12s - loss: 0.0302 - accuracy: 0.9895
 31/375 [=>............................] - ETA: 12s - loss: 0.0292 - accuracy: 0.9899
 33/375 [=>............................] - ETA: 12s - loss: 0.0304 - accuracy: 0.9898
 35/375 [=>............................] - ETA: 12s - loss: 0.0296 - accuracy: 0.9902
 37/375 [=>............................] - ETA: 12s - loss: 0.0286 - accuracy: 0.9905
 39/375 [==>...........................] - ETA: 12s - loss: 0.0274 - accuracy: 0.9910
 41/375 [==>...........................] - ETA: 12s - loss: 0.0270 - accuracy: 0.9910
 43/375 [==>...........................] - ETA: 12s - loss: 0.0279 - accuracy: 0.9911
 45/375 [==>...........................] - ETA: 11s - loss: 0.0289 - accuracy: 0.9906
 47/375 [==>...........................] - ETA: 11s - loss: 0.0288 - accuracy: 0.9905
 49/375 [==>...........................] - ETA: 11s - loss: 0.0292 - accuracy: 0.9904
 51/375 [===>..........................] - ETA: 11s - loss: 0.0294 - accuracy: 0.9900
 53/375 [===>..........................] - ETA: 11s - loss: 0.0294 - accuracy: 0.9898
 55/375 [===>..........................] - ETA: 11s - loss: 0.0293 - accuracy: 0.9898
 57/375 [===>..........................] - ETA: 11s - loss: 0.0298 - accuracy: 0.9896
 58/375 [===>..........................] - ETA: 11s - loss: 0.0297 - accuracy: 0.9896
 60/375 [===>..........................] - ETA: 11s - loss: 0.0292 - accuracy: 0.9898
 61/375 [===>..........................] - ETA: 11s - loss: 0.0293 - accuracy: 0.9898
 63/375 [====>.........................] - ETA: 11s - loss: 0.0302 - accuracy: 0.9898
 64/375 [====>.........................] - ETA: 11s - loss: 0.0299 - accuracy: 0.9900
 66/375 [====>.........................] - ETA: 11s - loss: 0.0292 - accuracy: 0.9903
 67/375 [====>.........................] - ETA: 11s - loss: 0.0292 - accuracy: 0.9903
 69/375 [====>.........................] - ETA: 12s - loss: 0.0287 - accuracy: 0.9905
 70/375 [====>.........................] - ETA: 12s - loss: 0.0288 - accuracy: 0.9904
 71/375 [====>.........................] - ETA: 12s - loss: 0.0286 - accuracy: 0.9905
 72/375 [====>.........................] - ETA: 12s - loss: 0.0285 - accuracy: 0.9905
 73/375 [====>.........................] - ETA: 12s - loss: 0.0294 - accuracy: 0.9904
 74/375 [====>.........................] - ETA: 12s - loss: 0.0290 - accuracy: 0.9905
 75/375 [=====>........................] - ETA: 12s - loss: 0.0288 - accuracy: 0.9906
 76/375 [=====>........................] - ETA: 12s - loss: 0.0288 - accuracy: 0.9905
 77/375 [=====>........................] - ETA: 13s - loss: 0.0291 - accuracy: 0.9904
 78/375 [=====>........................] - ETA: 13s - loss: 0.0299 - accuracy: 0.9903
 79/375 [=====>........................] - ETA: 13s - loss: 0.0298 - accuracy: 0.9902
 80/375 [=====>........................] - ETA: 13s - loss: 0.0299 - accuracy: 0.9902
 81/375 [=====>........................] - ETA: 13s - loss: 0.0298 - accuracy: 0.9903
 83/375 [=====>........................] - ETA: 13s - loss: 0.0307 - accuracy: 0.9898
 84/375 [=====>........................] - ETA: 13s - loss: 0.0304 - accuracy: 0.9900
 85/375 [=====>........................] - ETA: 13s - loss: 0.0305 - accuracy: 0.9900
 86/375 [=====>........................] - ETA: 13s - loss: 0.0308 - accuracy: 0.9899
 87/375 [=====>........................] - ETA: 13s - loss: 0.0315 - accuracy: 0.9899
 88/375 [======>.......................] - ETA: 13s - loss: 0.0321 - accuracy: 0.9898
 89/375 [======>.......................] - ETA: 13s - loss: 0.0323 - accuracy: 0.9896
 90/375 [======>.......................] - ETA: 13s - loss: 0.0326 - accuracy: 0.9894
 91/375 [======>.......................] - ETA: 13s - loss: 0.0325 - accuracy: 0.9894
 92/375 [======>.......................] - ETA: 13s - loss: 0.0327 - accuracy: 0.9894
 93/375 [======>.......................] - ETA: 13s - loss: 0.0324 - accuracy: 0.9895
 94/375 [======>.......................] - ETA: 13s - loss: 0.0323 - accuracy: 0.9894
 95/375 [======>.......................] - ETA: 13s - loss: 0.0322 - accuracy: 0.9895
 96/375 [======>.......................] - ETA: 13s - loss: 0.0319 - accuracy: 0.9896
 97/375 [======>.......................] - ETA: 13s - loss: 0.0317 - accuracy: 0.9897
 98/375 [======>.......................] - ETA: 13s - loss: 0.0314 - accuracy: 0.9898
 99/375 [======>.......................] - ETA: 13s - loss: 0.0313 - accuracy: 0.9897
100/375 [=======>......................] - ETA: 13s - loss: 0.0316 - accuracy: 0.9898
101/375 [=======>......................] - ETA: 13s - loss: 0.0317 - accuracy: 0.9896
102/375 [=======>......................] - ETA: 13s - loss: 0.0315 - accuracy: 0.9897
103/375 [=======>......................] - ETA: 13s - loss: 0.0314 - accuracy: 0.9896
104/375 [=======>......................] - ETA: 13s - loss: 0.0316 - accuracy: 0.9896
105/375 [=======>......................] - ETA: 13s - loss: 0.0320 - accuracy: 0.9895
107/375 [=======>......................] - ETA: 13s - loss: 0.0317 - accuracy: 0.9896
108/375 [=======>......................] - ETA: 13s - loss: 0.0314 - accuracy: 0.9897
109/375 [=======>......................] - ETA: 13s - loss: 0.0313 - accuracy: 0.9897
110/375 [=======>......................] - ETA: 13s - loss: 0.0312 - accuracy: 0.9898
112/375 [=======>......................] - ETA: 13s - loss: 0.0313 - accuracy: 0.9898
113/375 [========>.....................] - ETA: 13s - loss: 0.0313 - accuracy: 0.9898
114/375 [========>.....................] - ETA: 13s - loss: 0.0314 - accuracy: 0.9897
115/375 [========>.....................] - ETA: 13s - loss: 0.0316 - accuracy: 0.9897
116/375 [========>.....................] - ETA: 13s - loss: 0.0314 - accuracy: 0.9897
117/375 [========>.....................] - ETA: 13s - loss: 0.0314 - accuracy: 0.9897
118/375 [========>.....................] - ETA: 13s - loss: 0.0314 - accuracy: 0.9897
119/375 [========>.....................] - ETA: 13s - loss: 0.0313 - accuracy: 0.9897
120/375 [========>.....................] - ETA: 13s - loss: 0.0313 - accuracy: 0.9896
121/375 [========>.....................] - ETA: 13s - loss: 0.0314 - accuracy: 0.9896
122/375 [========>.....................] - ETA: 13s - loss: 0.0314 - accuracy: 0.9896
123/375 [========>.....................] - ETA: 13s - loss: 0.0317 - accuracy: 0.9895
125/375 [=========>....................] - ETA: 13s - loss: 0.0314 - accuracy: 0.9897
126/375 [=========>....................] - ETA: 13s - loss: 0.0313 - accuracy: 0.9896
127/375 [=========>....................] - ETA: 13s - loss: 0.0313 - accuracy: 0.9896
128/375 [=========>....................] - ETA: 13s - loss: 0.0311 - accuracy: 0.9897
129/375 [=========>....................] - ETA: 13s - loss: 0.0309 - accuracy: 0.9898
130/375 [=========>....................] - ETA: 13s - loss: 0.0309 - accuracy: 0.9897
131/375 [=========>....................] - ETA: 13s - loss: 0.0312 - accuracy: 0.9896
132/375 [=========>....................] - ETA: 12s - loss: 0.0310 - accuracy: 0.9896
133/375 [=========>....................] - ETA: 13s - loss: 0.0311 - accuracy: 0.9896
134/375 [=========>....................] - ETA: 13s - loss: 0.0311 - accuracy: 0.9896
136/375 [=========>....................] - ETA: 12s - loss: 0.0308 - accuracy: 0.9897
138/375 [==========>...................] - ETA: 12s - loss: 0.0304 - accuracy: 0.9898
139/375 [==========>...................] - ETA: 12s - loss: 0.0303 - accuracy: 0.9898
140/375 [==========>...................] - ETA: 12s - loss: 0.0302 - accuracy: 0.9899
141/375 [==========>...................] - ETA: 12s - loss: 0.0300 - accuracy: 0.9900
142/375 [==========>...................] - ETA: 12s - loss: 0.0301 - accuracy: 0.9899
143/375 [==========>...................] - ETA: 12s - loss: 0.0304 - accuracy: 0.9898
144/375 [==========>...................] - ETA: 12s - loss: 0.0304 - accuracy: 0.9898
145/375 [==========>...................] - ETA: 12s - loss: 0.0306 - accuracy: 0.9897
146/375 [==========>...................] - ETA: 12s - loss: 0.0308 - accuracy: 0.9896
147/375 [==========>...................] - ETA: 12s - loss: 0.0308 - accuracy: 0.9896
148/375 [==========>...................] - ETA: 12s - loss: 0.0309 - accuracy: 0.9896
149/375 [==========>...................] - ETA: 12s - loss: 0.0308 - accuracy: 0.9896
150/375 [===========>..................] - ETA: 12s - loss: 0.0307 - accuracy: 0.9896
151/375 [===========>..................] - ETA: 12s - loss: 0.0305 - accuracy: 0.9897
152/375 [===========>..................] - ETA: 12s - loss: 0.0308 - accuracy: 0.9896
153/375 [===========>..................] - ETA: 12s - loss: 0.0308 - accuracy: 0.9896
154/375 [===========>..................] - ETA: 12s - loss: 0.0309 - accuracy: 0.9896
155/375 [===========>..................] - ETA: 12s - loss: 0.0312 - accuracy: 0.9895
156/375 [===========>..................] - ETA: 12s - loss: 0.0311 - accuracy: 0.9895
157/375 [===========>..................] - ETA: 12s - loss: 0.0310 - accuracy: 0.9895
158/375 [===========>..................] - ETA: 12s - loss: 0.0313 - accuracy: 0.9895
159/375 [===========>..................] - ETA: 12s - loss: 0.0313 - accuracy: 0.9895
160/375 [===========>..................] - ETA: 12s - loss: 0.0312 - accuracy: 0.9895
161/375 [===========>..................] - ETA: 12s - loss: 0.0312 - accuracy: 0.9895
163/375 [============>.................] - ETA: 11s - loss: 0.0315 - accuracy: 0.9895
165/375 [============>.................] - ETA: 11s - loss: 0.0313 - accuracy: 0.9895
166/375 [============>.................] - ETA: 11s - loss: 0.0315 - accuracy: 0.9895
167/375 [============>.................] - ETA: 11s - loss: 0.0314 - accuracy: 0.9895
168/375 [============>.................] - ETA: 11s - loss: 0.0314 - accuracy: 0.9895
169/375 [============>.................] - ETA: 11s - loss: 0.0314 - accuracy: 0.9895
170/375 [============>.................] - ETA: 11s - loss: 0.0315 - accuracy: 0.9894
172/375 [============>.................] - ETA: 11s - loss: 0.0318 - accuracy: 0.9894
173/375 [============>.................] - ETA: 11s - loss: 0.0316 - accuracy: 0.9895
174/375 [============>.................] - ETA: 11s - loss: 0.0315 - accuracy: 0.9895
175/375 [=============>................] - ETA: 11s - loss: 0.0313 - accuracy: 0.9896
176/375 [=============>................] - ETA: 11s - loss: 0.0312 - accuracy: 0.9896
177/375 [=============>................] - ETA: 11s - loss: 0.0313 - accuracy: 0.9896
178/375 [=============>................] - ETA: 11s - loss: 0.0312 - accuracy: 0.9896
179/375 [=============>................] - ETA: 11s - loss: 0.0312 - accuracy: 0.9896
180/375 [=============>................] - ETA: 11s - loss: 0.0311 - accuracy: 0.9896
181/375 [=============>................] - ETA: 11s - loss: 0.0310 - accuracy: 0.9896
182/375 [=============>................] - ETA: 11s - loss: 0.0309 - accuracy: 0.9897
183/375 [=============>................] - ETA: 11s - loss: 0.0309 - accuracy: 0.9897
184/375 [=============>................] - ETA: 10s - loss: 0.0307 - accuracy: 0.9898
185/375 [=============>................] - ETA: 10s - loss: 0.0307 - accuracy: 0.9898
186/375 [=============>................] - ETA: 10s - loss: 0.0307 - accuracy: 0.9898
187/375 [=============>................] - ETA: 10s - loss: 0.0307 - accuracy: 0.9898
188/375 [==============>...............] - ETA: 10s - loss: 0.0308 - accuracy: 0.9898
189/375 [==============>...............] - ETA: 10s - loss: 0.0308 - accuracy: 0.9898
190/375 [==============>...............] - ETA: 10s - loss: 0.0307 - accuracy: 0.9898
191/375 [==============>...............] - ETA: 10s - loss: 0.0308 - accuracy: 0.9897
192/375 [==============>...............] - ETA: 10s - loss: 0.0308 - accuracy: 0.9897
193/375 [==============>...............] - ETA: 10s - loss: 0.0307 - accuracy: 0.9897
194/375 [==============>...............] - ETA: 10s - loss: 0.0308 - accuracy: 0.9897
195/375 [==============>...............] - ETA: 10s - loss: 0.0310 - accuracy: 0.9897
196/375 [==============>...............] - ETA: 10s - loss: 0.0309 - accuracy: 0.9897
197/375 [==============>...............] - ETA: 10s - loss: 0.0307 - accuracy: 0.9898
198/375 [==============>...............] - ETA: 10s - loss: 0.0307 - accuracy: 0.9898
199/375 [==============>...............] - ETA: 10s - loss: 0.0308 - accuracy: 0.9898
200/375 [===============>..............] - ETA: 10s - loss: 0.0309 - accuracy: 0.9898
201/375 [===============>..............] - ETA: 10s - loss: 0.0310 - accuracy: 0.9898
202/375 [===============>..............] - ETA: 10s - loss: 0.0309 - accuracy: 0.9898
203/375 [===============>..............] - ETA: 10s - loss: 0.0309 - accuracy: 0.9898
204/375 [===============>..............] - ETA: 10s - loss: 0.0308 - accuracy: 0.9898
205/375 [===============>..............] - ETA: 10s - loss: 0.0308 - accuracy: 0.9898
206/375 [===============>..............] - ETA: 10s - loss: 0.0309 - accuracy: 0.9898
207/375 [===============>..............] - ETA: 9s - loss: 0.0311 - accuracy: 0.9897 
208/375 [===============>..............] - ETA: 9s - loss: 0.0311 - accuracy: 0.9896
209/375 [===============>..............] - ETA: 9s - loss: 0.0310 - accuracy: 0.9896
210/375 [===============>..............] - ETA: 9s - loss: 0.0309 - accuracy: 0.9897
212/375 [===============>..............] - ETA: 9s - loss: 0.0311 - accuracy: 0.9896
213/375 [================>.............] - ETA: 9s - loss: 0.0310 - accuracy: 0.9896
214/375 [================>.............] - ETA: 9s - loss: 0.0311 - accuracy: 0.9896
215/375 [================>.............] - ETA: 9s - loss: 0.0310 - accuracy: 0.9896
217/375 [================>.............] - ETA: 9s - loss: 0.0310 - accuracy: 0.9896
219/375 [================>.............] - ETA: 9s - loss: 0.0309 - accuracy: 0.9896
221/375 [================>.............] - ETA: 9s - loss: 0.0309 - accuracy: 0.9896
222/375 [================>.............] - ETA: 9s - loss: 0.0308 - accuracy: 0.9896
223/375 [================>.............] - ETA: 9s - loss: 0.0307 - accuracy: 0.9896
224/375 [================>.............] - ETA: 8s - loss: 0.0307 - accuracy: 0.9896
225/375 [=================>............] - ETA: 8s - loss: 0.0306 - accuracy: 0.9896
227/375 [=================>............] - ETA: 8s - loss: 0.0305 - accuracy: 0.9896
229/375 [=================>............] - ETA: 8s - loss: 0.0310 - accuracy: 0.9895
230/375 [=================>............] - ETA: 8s - loss: 0.0310 - accuracy: 0.9895
232/375 [=================>............] - ETA: 8s - loss: 0.0313 - accuracy: 0.9895
234/375 [=================>............] - ETA: 8s - loss: 0.0313 - accuracy: 0.9894
236/375 [=================>............] - ETA: 8s - loss: 0.0313 - accuracy: 0.9894
238/375 [==================>...........] - ETA: 8s - loss: 0.0313 - accuracy: 0.9895
239/375 [==================>...........] - ETA: 8s - loss: 0.0313 - accuracy: 0.9894
241/375 [==================>...........] - ETA: 7s - loss: 0.0310 - accuracy: 0.9895
242/375 [==================>...........] - ETA: 7s - loss: 0.0310 - accuracy: 0.9895
243/375 [==================>...........] - ETA: 7s - loss: 0.0310 - accuracy: 0.9895
245/375 [==================>...........] - ETA: 7s - loss: 0.0309 - accuracy: 0.9895
247/375 [==================>...........] - ETA: 7s - loss: 0.0308 - accuracy: 0.9896
249/375 [==================>...........] - ETA: 7s - loss: 0.0311 - accuracy: 0.9896
250/375 [===================>..........] - ETA: 7s - loss: 0.0310 - accuracy: 0.9896
251/375 [===================>..........] - ETA: 7s - loss: 0.0309 - accuracy: 0.9896
252/375 [===================>..........] - ETA: 7s - loss: 0.0312 - accuracy: 0.9895
253/375 [===================>..........] - ETA: 7s - loss: 0.0311 - accuracy: 0.9895
255/375 [===================>..........] - ETA: 7s - loss: 0.0310 - accuracy: 0.9896
256/375 [===================>..........] - ETA: 6s - loss: 0.0310 - accuracy: 0.9896
258/375 [===================>..........] - ETA: 6s - loss: 0.0309 - accuracy: 0.9896
259/375 [===================>..........] - ETA: 6s - loss: 0.0309 - accuracy: 0.9896
260/375 [===================>..........] - ETA: 6s - loss: 0.0311 - accuracy: 0.9895
261/375 [===================>..........] - ETA: 6s - loss: 0.0310 - accuracy: 0.9895
262/375 [===================>..........] - ETA: 6s - loss: 0.0311 - accuracy: 0.9895
263/375 [====================>.........] - ETA: 6s - loss: 0.0313 - accuracy: 0.9895
264/375 [====================>.........] - ETA: 6s - loss: 0.0312 - accuracy: 0.9895
265/375 [====================>.........] - ETA: 6s - loss: 0.0313 - accuracy: 0.9895
266/375 [====================>.........] - ETA: 6s - loss: 0.0312 - accuracy: 0.9895
267/375 [====================>.........] - ETA: 6s - loss: 0.0311 - accuracy: 0.9896
268/375 [====================>.........] - ETA: 6s - loss: 0.0312 - accuracy: 0.9895
269/375 [====================>.........] - ETA: 6s - loss: 0.0311 - accuracy: 0.9895
270/375 [====================>.........] - ETA: 6s - loss: 0.0310 - accuracy: 0.9896
271/375 [====================>.........] - ETA: 6s - loss: 0.0309 - accuracy: 0.9896
272/375 [====================>.........] - ETA: 6s - loss: 0.0309 - accuracy: 0.9896
273/375 [====================>.........] - ETA: 6s - loss: 0.0308 - accuracy: 0.9896
274/375 [====================>.........] - ETA: 5s - loss: 0.0307 - accuracy: 0.9897
275/375 [=====================>........] - ETA: 5s - loss: 0.0306 - accuracy: 0.9897
276/375 [=====================>........] - ETA: 5s - loss: 0.0306 - accuracy: 0.9897
277/375 [=====================>........] - ETA: 5s - loss: 0.0305 - accuracy: 0.9898
278/375 [=====================>........] - ETA: 5s - loss: 0.0304 - accuracy: 0.9898
279/375 [=====================>........] - ETA: 5s - loss: 0.0304 - accuracy: 0.9898
280/375 [=====================>........] - ETA: 5s - loss: 0.0303 - accuracy: 0.9898
282/375 [=====================>........] - ETA: 5s - loss: 0.0303 - accuracy: 0.9898
284/375 [=====================>........] - ETA: 5s - loss: 0.0303 - accuracy: 0.9898
286/375 [=====================>........] - ETA: 5s - loss: 0.0302 - accuracy: 0.9898
288/375 [======================>.......] - ETA: 5s - loss: 0.0302 - accuracy: 0.9898
289/375 [======================>.......] - ETA: 5s - loss: 0.0303 - accuracy: 0.9898
290/375 [======================>.......] - ETA: 5s - loss: 0.0306 - accuracy: 0.9896
291/375 [======================>.......] - ETA: 4s - loss: 0.0305 - accuracy: 0.9896
293/375 [======================>.......] - ETA: 4s - loss: 0.0306 - accuracy: 0.9896
295/375 [======================>.......] - ETA: 4s - loss: 0.0305 - accuracy: 0.9896
296/375 [======================>.......] - ETA: 4s - loss: 0.0304 - accuracy: 0.9896
297/375 [======================>.......] - ETA: 4s - loss: 0.0305 - accuracy: 0.9896
298/375 [======================>.......] - ETA: 4s - loss: 0.0305 - accuracy: 0.9896
299/375 [======================>.......] - ETA: 4s - loss: 0.0305 - accuracy: 0.9896
301/375 [=======================>......] - ETA: 4s - loss: 0.0305 - accuracy: 0.9896
303/375 [=======================>......] - ETA: 4s - loss: 0.0304 - accuracy: 0.9896
305/375 [=======================>......] - ETA: 4s - loss: 0.0306 - accuracy: 0.9896
307/375 [=======================>......] - ETA: 3s - loss: 0.0306 - accuracy: 0.9896
308/375 [=======================>......] - ETA: 3s - loss: 0.0305 - accuracy: 0.9896
310/375 [=======================>......] - ETA: 3s - loss: 0.0305 - accuracy: 0.9896
311/375 [=======================>......] - ETA: 3s - loss: 0.0305 - accuracy: 0.9896
312/375 [=======================>......] - ETA: 3s - loss: 0.0305 - accuracy: 0.9896
313/375 [========================>.....] - ETA: 3s - loss: 0.0306 - accuracy: 0.9896
314/375 [========================>.....] - ETA: 3s - loss: 0.0305 - accuracy: 0.9896
316/375 [========================>.....] - ETA: 3s - loss: 0.0306 - accuracy: 0.9896
318/375 [========================>.....] - ETA: 3s - loss: 0.0305 - accuracy: 0.9896
320/375 [========================>.....] - ETA: 3s - loss: 0.0305 - accuracy: 0.9896
322/375 [========================>.....] - ETA: 3s - loss: 0.0304 - accuracy: 0.9897
324/375 [========================>.....] - ETA: 2s - loss: 0.0303 - accuracy: 0.9897
325/375 [=========================>....] - ETA: 2s - loss: 0.0303 - accuracy: 0.9897
326/375 [=========================>....] - ETA: 2s - loss: 0.0302 - accuracy: 0.9897
327/375 [=========================>....] - ETA: 2s - loss: 0.0302 - accuracy: 0.9897
329/375 [=========================>....] - ETA: 2s - loss: 0.0301 - accuracy: 0.9897
330/375 [=========================>....] - ETA: 2s - loss: 0.0300 - accuracy: 0.9898
331/375 [=========================>....] - ETA: 2s - loss: 0.0300 - accuracy: 0.9898
332/375 [=========================>....] - ETA: 2s - loss: 0.0300 - accuracy: 0.9898
333/375 [=========================>....] - ETA: 2s - loss: 0.0299 - accuracy: 0.9898
334/375 [=========================>....] - ETA: 2s - loss: 0.0298 - accuracy: 0.9898
335/375 [=========================>....] - ETA: 2s - loss: 0.0298 - accuracy: 0.9899
336/375 [=========================>....] - ETA: 2s - loss: 0.0299 - accuracy: 0.9898
337/375 [=========================>....] - ETA: 2s - loss: 0.0298 - accuracy: 0.9898
338/375 [==========================>...] - ETA: 2s - loss: 0.0298 - accuracy: 0.9899
339/375 [==========================>...] - ETA: 2s - loss: 0.0297 - accuracy: 0.9899
340/375 [==========================>...] - ETA: 2s - loss: 0.0298 - accuracy: 0.9898
341/375 [==========================>...] - ETA: 2s - loss: 0.0297 - accuracy: 0.9899
342/375 [==========================>...] - ETA: 1s - loss: 0.0296 - accuracy: 0.9899
343/375 [==========================>...] - ETA: 1s - loss: 0.0296 - accuracy: 0.9899
344/375 [==========================>...] - ETA: 1s - loss: 0.0296 - accuracy: 0.9899
345/375 [==========================>...] - ETA: 1s - loss: 0.0295 - accuracy: 0.9899
346/375 [==========================>...] - ETA: 1s - loss: 0.0295 - accuracy: 0.9899
347/375 [==========================>...] - ETA: 1s - loss: 0.0297 - accuracy: 0.9899
348/375 [==========================>...] - ETA: 1s - loss: 0.0297 - accuracy: 0.9899
349/375 [==========================>...] - ETA: 1s - loss: 0.0297 - accuracy: 0.9899
350/375 [===========================>..] - ETA: 1s - loss: 0.0297 - accuracy: 0.9899
351/375 [===========================>..] - ETA: 1s - loss: 0.0296 - accuracy: 0.9899
352/375 [===========================>..] - ETA: 1s - loss: 0.0296 - accuracy: 0.9899
353/375 [===========================>..] - ETA: 1s - loss: 0.0296 - accuracy: 0.9899
354/375 [===========================>..] - ETA: 1s - loss: 0.0297 - accuracy: 0.9899
355/375 [===========================>..] - ETA: 1s - loss: 0.0296 - accuracy: 0.9899
356/375 [===========================>..] - ETA: 1s - loss: 0.0297 - accuracy: 0.9899
357/375 [===========================>..] - ETA: 1s - loss: 0.0299 - accuracy: 0.9898
358/375 [===========================>..] - ETA: 1s - loss: 0.0298 - accuracy: 0.9899
360/375 [===========================>..] - ETA: 0s - loss: 0.0304 - accuracy: 0.9897
361/375 [===========================>..] - ETA: 0s - loss: 0.0303 - accuracy: 0.9897
362/375 [===========================>..] - ETA: 0s - loss: 0.0303 - accuracy: 0.9897
363/375 [============================>.] - ETA: 0s - loss: 0.0302 - accuracy: 0.9897
364/375 [============================>.] - ETA: 0s - loss: 0.0302 - accuracy: 0.9898
365/375 [============================>.] - ETA: 0s - loss: 0.0301 - accuracy: 0.9898
366/375 [============================>.] - ETA: 0s - loss: 0.0301 - accuracy: 0.9898
367/375 [============================>.] - ETA: 0s - loss: 0.0302 - accuracy: 0.9897
368/375 [============================>.] - ETA: 0s - loss: 0.0302 - accuracy: 0.9898
369/375 [============================>.] - ETA: 0s - loss: 0.0303 - accuracy: 0.9897
370/375 [============================>.] - ETA: 0s - loss: 0.0303 - accuracy: 0.9897
371/375 [============================>.] - ETA: 0s - loss: 0.0302 - accuracy: 0.9898
372/375 [============================>.] - ETA: 0s - loss: 0.0301 - accuracy: 0.9898
373/375 [============================>.] - ETA: 0s - loss: 0.0301 - accuracy: 0.9898
374/375 [============================>.] - ETA: 0s - loss: 0.0303 - accuracy: 0.9897
375/375 [==============================] - 22s 60ms/step - loss: 0.0303 - accuracy: 0.9897

375/375 [==============================] - 24s 64ms/step - loss: 0.0303 - accuracy: 0.9897 - val_loss: 0.0461 - val_accuracy: 0.9866
Epoch 5/5

  1/375 [..............................] - ETA: 10s - loss: 0.0365 - accuracy: 0.9844
  3/375 [..............................] - ETA: 12s - loss: 0.0235 - accuracy: 0.9896
  5/375 [..............................] - ETA: 12s - loss: 0.0216 - accuracy: 0.9906
  7/375 [..............................] - ETA: 12s - loss: 0.0222 - accuracy: 0.9911
  9/375 [..............................] - ETA: 12s - loss: 0.0207 - accuracy: 0.9922
 11/375 [..............................] - ETA: 12s - loss: 0.0211 - accuracy: 0.9915
 13/375 [>.............................] - ETA: 12s - loss: 0.0210 - accuracy: 0.9916
 15/375 [>.............................] - ETA: 12s - loss: 0.0196 - accuracy: 0.9922
 17/375 [>.............................] - ETA: 12s - loss: 0.0187 - accuracy: 0.9926
 19/375 [>.............................] - ETA: 12s - loss: 0.0179 - accuracy: 0.9930
 21/375 [>.............................] - ETA: 11s - loss: 0.0166 - accuracy: 0.9937
 23/375 [>.............................] - ETA: 11s - loss: 0.0170 - accuracy: 0.9939
 25/375 [=>............................] - ETA: 11s - loss: 0.0178 - accuracy: 0.9937
 27/375 [=>............................] - ETA: 11s - loss: 0.0188 - accuracy: 0.9939
 29/375 [=>............................] - ETA: 11s - loss: 0.0187 - accuracy: 0.9941
 31/375 [=>............................] - ETA: 11s - loss: 0.0176 - accuracy: 0.9945
 33/375 [=>............................] - ETA: 11s - loss: 0.0166 - accuracy: 0.9948
 35/375 [=>............................] - ETA: 11s - loss: 0.0184 - accuracy: 0.9949
 37/375 [=>............................] - ETA: 11s - loss: 0.0213 - accuracy: 0.9937
 39/375 [==>...........................] - ETA: 11s - loss: 0.0219 - accuracy: 0.9936
 41/375 [==>...........................] - ETA: 11s - loss: 0.0210 - accuracy: 0.9939
 43/375 [==>...........................] - ETA: 11s - loss: 0.0209 - accuracy: 0.9940
 44/375 [==>...........................] - ETA: 11s - loss: 0.0205 - accuracy: 0.9941
 46/375 [==>...........................] - ETA: 11s - loss: 0.0206 - accuracy: 0.9941
 48/375 [==>...........................] - ETA: 11s - loss: 0.0212 - accuracy: 0.9937
 50/375 [===>..........................] - ETA: 11s - loss: 0.0207 - accuracy: 0.9939
 52/375 [===>..........................] - ETA: 11s - loss: 0.0205 - accuracy: 0.9940
 53/375 [===>..........................] - ETA: 11s - loss: 0.0203 - accuracy: 0.9941
 55/375 [===>..........................] - ETA: 11s - loss: 0.0202 - accuracy: 0.9939
 57/375 [===>..........................] - ETA: 11s - loss: 0.0210 - accuracy: 0.9937
 59/375 [===>..........................] - ETA: 11s - loss: 0.0207 - accuracy: 0.9938
 61/375 [===>..........................] - ETA: 11s - loss: 0.0204 - accuracy: 0.9940
 63/375 [====>.........................] - ETA: 11s - loss: 0.0202 - accuracy: 0.9940
 64/375 [====>.........................] - ETA: 11s - loss: 0.0204 - accuracy: 0.9940
 65/375 [====>.........................] - ETA: 11s - loss: 0.0204 - accuracy: 0.9940
 66/375 [====>.........................] - ETA: 11s - loss: 0.0211 - accuracy: 0.9940
 67/375 [====>.........................] - ETA: 11s - loss: 0.0214 - accuracy: 0.9939
 68/375 [====>.........................] - ETA: 11s - loss: 0.0217 - accuracy: 0.9937
 69/375 [====>.........................] - ETA: 11s - loss: 0.0219 - accuracy: 0.9937
 70/375 [====>.........................] - ETA: 11s - loss: 0.0219 - accuracy: 0.9936
 71/375 [====>.........................] - ETA: 11s - loss: 0.0221 - accuracy: 0.9936
 73/375 [====>.........................] - ETA: 11s - loss: 0.0227 - accuracy: 0.9934
 74/375 [====>.........................] - ETA: 11s - loss: 0.0230 - accuracy: 0.9932
 75/375 [=====>........................] - ETA: 12s - loss: 0.0230 - accuracy: 0.9932
 76/375 [=====>........................] - ETA: 12s - loss: 0.0230 - accuracy: 0.9932
 77/375 [=====>........................] - ETA: 12s - loss: 0.0228 - accuracy: 0.9933
 78/375 [=====>........................] - ETA: 12s - loss: 0.0228 - accuracy: 0.9932
 79/375 [=====>........................] - ETA: 12s - loss: 0.0226 - accuracy: 0.9933
 80/375 [=====>........................] - ETA: 12s - loss: 0.0226 - accuracy: 0.9932
 81/375 [=====>........................] - ETA: 12s - loss: 0.0227 - accuracy: 0.9931
 82/375 [=====>........................] - ETA: 12s - loss: 0.0225 - accuracy: 0.9931
 83/375 [=====>........................] - ETA: 12s - loss: 0.0222 - accuracy: 0.9932
 84/375 [=====>........................] - ETA: 12s - loss: 0.0224 - accuracy: 0.9931
 85/375 [=====>........................] - ETA: 12s - loss: 0.0221 - accuracy: 0.9932
 86/375 [=====>........................] - ETA: 12s - loss: 0.0220 - accuracy: 0.9932
 87/375 [=====>........................] - ETA: 12s - loss: 0.0218 - accuracy: 0.9933
 88/375 [======>.......................] - ETA: 12s - loss: 0.0217 - accuracy: 0.9933
 89/375 [======>.......................] - ETA: 12s - loss: 0.0217 - accuracy: 0.9933
 90/375 [======>.......................] - ETA: 12s - loss: 0.0215 - accuracy: 0.9934
 91/375 [======>.......................] - ETA: 12s - loss: 0.0213 - accuracy: 0.9935
 92/375 [======>.......................] - ETA: 12s - loss: 0.0217 - accuracy: 0.9933
 93/375 [======>.......................] - ETA: 13s - loss: 0.0216 - accuracy: 0.9934
 94/375 [======>.......................] - ETA: 13s - loss: 0.0215 - accuracy: 0.9934
 95/375 [======>.......................] - ETA: 13s - loss: 0.0217 - accuracy: 0.9933
 97/375 [======>.......................] - ETA: 13s - loss: 0.0220 - accuracy: 0.9932
 98/375 [======>.......................] - ETA: 13s - loss: 0.0219 - accuracy: 0.9931
 99/375 [======>.......................] - ETA: 13s - loss: 0.0219 - accuracy: 0.9931
100/375 [=======>......................] - ETA: 13s - loss: 0.0221 - accuracy: 0.9930
101/375 [=======>......................] - ETA: 13s - loss: 0.0221 - accuracy: 0.9930
102/375 [=======>......................] - ETA: 13s - loss: 0.0221 - accuracy: 0.9930
103/375 [=======>......................] - ETA: 13s - loss: 0.0220 - accuracy: 0.9929
104/375 [=======>......................] - ETA: 13s - loss: 0.0220 - accuracy: 0.9929
105/375 [=======>......................] - ETA: 13s - loss: 0.0218 - accuracy: 0.9930
106/375 [=======>......................] - ETA: 13s - loss: 0.0219 - accuracy: 0.9930
107/375 [=======>......................] - ETA: 13s - loss: 0.0219 - accuracy: 0.9930
108/375 [=======>......................] - ETA: 13s - loss: 0.0217 - accuracy: 0.9931
110/375 [=======>......................] - ETA: 13s - loss: 0.0220 - accuracy: 0.9928
111/375 [=======>......................] - ETA: 13s - loss: 0.0219 - accuracy: 0.9928
113/375 [========>.....................] - ETA: 13s - loss: 0.0217 - accuracy: 0.9929
114/375 [========>.....................] - ETA: 13s - loss: 0.0216 - accuracy: 0.9929
115/375 [========>.....................] - ETA: 13s - loss: 0.0214 - accuracy: 0.9930
116/375 [========>.....................] - ETA: 13s - loss: 0.0225 - accuracy: 0.9928
117/375 [========>.....................] - ETA: 13s - loss: 0.0225 - accuracy: 0.9928
118/375 [========>.....................] - ETA: 13s - loss: 0.0224 - accuracy: 0.9928
119/375 [========>.....................] - ETA: 13s - loss: 0.0222 - accuracy: 0.9929
121/375 [========>.....................] - ETA: 13s - loss: 0.0227 - accuracy: 0.9929
122/375 [========>.....................] - ETA: 13s - loss: 0.0231 - accuracy: 0.9929
123/375 [========>.....................] - ETA: 13s - loss: 0.0232 - accuracy: 0.9928
125/375 [=========>....................] - ETA: 13s - loss: 0.0231 - accuracy: 0.9928
126/375 [=========>....................] - ETA: 13s - loss: 0.0230 - accuracy: 0.9929
127/375 [=========>....................] - ETA: 13s - loss: 0.0231 - accuracy: 0.9928
128/375 [=========>....................] - ETA: 13s - loss: 0.0230 - accuracy: 0.9928
130/375 [=========>....................] - ETA: 13s - loss: 0.0232 - accuracy: 0.9927
132/375 [=========>....................] - ETA: 12s - loss: 0.0235 - accuracy: 0.9928
133/375 [=========>....................] - ETA: 12s - loss: 0.0237 - accuracy: 0.9928
134/375 [=========>....................] - ETA: 12s - loss: 0.0236 - accuracy: 0.9928
135/375 [=========>....................] - ETA: 12s - loss: 0.0234 - accuracy: 0.9928
136/375 [=========>....................] - ETA: 12s - loss: 0.0239 - accuracy: 0.9927
137/375 [=========>....................] - ETA: 12s - loss: 0.0239 - accuracy: 0.9926
138/375 [==========>...................] - ETA: 12s - loss: 0.0238 - accuracy: 0.9927
140/375 [==========>...................] - ETA: 12s - loss: 0.0236 - accuracy: 0.9927
141/375 [==========>...................] - ETA: 12s - loss: 0.0235 - accuracy: 0.9927
142/375 [==========>...................] - ETA: 12s - loss: 0.0235 - accuracy: 0.9927
143/375 [==========>...................] - ETA: 12s - loss: 0.0234 - accuracy: 0.9927
145/375 [==========>...................] - ETA: 12s - loss: 0.0241 - accuracy: 0.9927
146/375 [==========>...................] - ETA: 12s - loss: 0.0242 - accuracy: 0.9926
147/375 [==========>...................] - ETA: 12s - loss: 0.0240 - accuracy: 0.9927
148/375 [==========>...................] - ETA: 12s - loss: 0.0240 - accuracy: 0.9927
149/375 [==========>...................] - ETA: 12s - loss: 0.0239 - accuracy: 0.9928
151/375 [===========>..................] - ETA: 12s - loss: 0.0240 - accuracy: 0.9928
152/375 [===========>..................] - ETA: 12s - loss: 0.0239 - accuracy: 0.9928
153/375 [===========>..................] - ETA: 12s - loss: 0.0238 - accuracy: 0.9929
154/375 [===========>..................] - ETA: 12s - loss: 0.0237 - accuracy: 0.9929
155/375 [===========>..................] - ETA: 12s - loss: 0.0236 - accuracy: 0.9929
156/375 [===========>..................] - ETA: 12s - loss: 0.0236 - accuracy: 0.9929
157/375 [===========>..................] - ETA: 12s - loss: 0.0236 - accuracy: 0.9929
158/375 [===========>..................] - ETA: 11s - loss: 0.0235 - accuracy: 0.9929
159/375 [===========>..................] - ETA: 11s - loss: 0.0236 - accuracy: 0.9929
161/375 [===========>..................] - ETA: 11s - loss: 0.0235 - accuracy: 0.9929
162/375 [===========>..................] - ETA: 11s - loss: 0.0234 - accuracy: 0.9929
163/375 [============>.................] - ETA: 11s - loss: 0.0235 - accuracy: 0.9929
164/375 [============>.................] - ETA: 11s - loss: 0.0234 - accuracy: 0.9929
165/375 [============>.................] - ETA: 11s - loss: 0.0235 - accuracy: 0.9929
166/375 [============>.................] - ETA: 11s - loss: 0.0235 - accuracy: 0.9928
167/375 [============>.................] - ETA: 11s - loss: 0.0237 - accuracy: 0.9928
168/375 [============>.................] - ETA: 11s - loss: 0.0237 - accuracy: 0.9928
169/375 [============>.................] - ETA: 11s - loss: 0.0236 - accuracy: 0.9928
170/375 [============>.................] - ETA: 11s - loss: 0.0236 - accuracy: 0.9928
171/375 [============>.................] - ETA: 11s - loss: 0.0235 - accuracy: 0.9929
172/375 [============>.................] - ETA: 11s - loss: 0.0235 - accuracy: 0.9928
173/375 [============>.................] - ETA: 11s - loss: 0.0237 - accuracy: 0.9928
174/375 [============>.................] - ETA: 11s - loss: 0.0236 - accuracy: 0.9929
175/375 [=============>................] - ETA: 11s - loss: 0.0236 - accuracy: 0.9929
176/375 [=============>................] - ETA: 11s - loss: 0.0237 - accuracy: 0.9929
177/375 [=============>................] - ETA: 11s - loss: 0.0238 - accuracy: 0.9928
178/375 [=============>................] - ETA: 11s - loss: 0.0237 - accuracy: 0.9928
179/375 [=============>................] - ETA: 11s - loss: 0.0239 - accuracy: 0.9928
180/375 [=============>................] - ETA: 11s - loss: 0.0238 - accuracy: 0.9929
181/375 [=============>................] - ETA: 11s - loss: 0.0237 - accuracy: 0.9929
182/375 [=============>................] - ETA: 11s - loss: 0.0236 - accuracy: 0.9930
183/375 [=============>................] - ETA: 11s - loss: 0.0238 - accuracy: 0.9930
184/375 [=============>................] - ETA: 11s - loss: 0.0237 - accuracy: 0.9930
185/375 [=============>................] - ETA: 11s - loss: 0.0241 - accuracy: 0.9929
186/375 [=============>................] - ETA: 11s - loss: 0.0241 - accuracy: 0.9928
187/375 [=============>................] - ETA: 11s - loss: 0.0240 - accuracy: 0.9929
188/375 [==============>...............] - ETA: 11s - loss: 0.0240 - accuracy: 0.9929
189/375 [==============>...............] - ETA: 10s - loss: 0.0240 - accuracy: 0.9928
190/375 [==============>...............] - ETA: 10s - loss: 0.0239 - accuracy: 0.9929
191/375 [==============>...............] - ETA: 10s - loss: 0.0238 - accuracy: 0.9929
192/375 [==============>...............] - ETA: 10s - loss: 0.0237 - accuracy: 0.9929
193/375 [==============>...............] - ETA: 10s - loss: 0.0240 - accuracy: 0.9928
194/375 [==============>...............] - ETA: 10s - loss: 0.0239 - accuracy: 0.9929
195/375 [==============>...............] - ETA: 10s - loss: 0.0240 - accuracy: 0.9928
196/375 [==============>...............] - ETA: 10s - loss: 0.0240 - accuracy: 0.9928
197/375 [==============>...............] - ETA: 10s - loss: 0.0239 - accuracy: 0.9928
198/375 [==============>...............] - ETA: 10s - loss: 0.0239 - accuracy: 0.9928
199/375 [==============>...............] - ETA: 10s - loss: 0.0238 - accuracy: 0.9929
200/375 [===============>..............] - ETA: 10s - loss: 0.0237 - accuracy: 0.9929
201/375 [===============>..............] - ETA: 10s - loss: 0.0237 - accuracy: 0.9928
203/375 [===============>..............] - ETA: 10s - loss: 0.0239 - accuracy: 0.9927
204/375 [===============>..............] - ETA: 10s - loss: 0.0240 - accuracy: 0.9926
205/375 [===============>..............] - ETA: 10s - loss: 0.0241 - accuracy: 0.9926
207/375 [===============>..............] - ETA: 10s - loss: 0.0243 - accuracy: 0.9925
208/375 [===============>..............] - ETA: 10s - loss: 0.0244 - accuracy: 0.9925
209/375 [===============>..............] - ETA: 10s - loss: 0.0244 - accuracy: 0.9925
211/375 [===============>..............] - ETA: 9s - loss: 0.0243 - accuracy: 0.9926 
212/375 [===============>..............] - ETA: 9s - loss: 0.0242 - accuracy: 0.9926
213/375 [================>.............] - ETA: 9s - loss: 0.0241 - accuracy: 0.9926
214/375 [================>.............] - ETA: 9s - loss: 0.0241 - accuracy: 0.9926
215/375 [================>.............] - ETA: 9s - loss: 0.0241 - accuracy: 0.9926
216/375 [================>.............] - ETA: 9s - loss: 0.0241 - accuracy: 0.9926
217/375 [================>.............] - ETA: 9s - loss: 0.0241 - accuracy: 0.9926
218/375 [================>.............] - ETA: 9s - loss: 0.0241 - accuracy: 0.9926
220/375 [================>.............] - ETA: 9s - loss: 0.0239 - accuracy: 0.9926
221/375 [================>.............] - ETA: 9s - loss: 0.0239 - accuracy: 0.9926
223/375 [================>.............] - ETA: 9s - loss: 0.0243 - accuracy: 0.9926
224/375 [================>.............] - ETA: 9s - loss: 0.0242 - accuracy: 0.9925
225/375 [=================>............] - ETA: 9s - loss: 0.0242 - accuracy: 0.9926
226/375 [=================>............] - ETA: 8s - loss: 0.0241 - accuracy: 0.9926
227/375 [=================>............] - ETA: 8s - loss: 0.0241 - accuracy: 0.9926
228/375 [=================>............] - ETA: 8s - loss: 0.0240 - accuracy: 0.9927
229/375 [=================>............] - ETA: 8s - loss: 0.0239 - accuracy: 0.9927
230/375 [=================>............] - ETA: 8s - loss: 0.0239 - accuracy: 0.9927
231/375 [=================>............] - ETA: 8s - loss: 0.0240 - accuracy: 0.9926
232/375 [=================>............] - ETA: 8s - loss: 0.0242 - accuracy: 0.9926
233/375 [=================>............] - ETA: 8s - loss: 0.0242 - accuracy: 0.9926
234/375 [=================>............] - ETA: 8s - loss: 0.0244 - accuracy: 0.9925
235/375 [=================>............] - ETA: 8s - loss: 0.0244 - accuracy: 0.9924
237/375 [=================>............] - ETA: 8s - loss: 0.0243 - accuracy: 0.9925
238/375 [==================>...........] - ETA: 8s - loss: 0.0244 - accuracy: 0.9925
239/375 [==================>...........] - ETA: 8s - loss: 0.0244 - accuracy: 0.9925
240/375 [==================>...........] - ETA: 8s - loss: 0.0244 - accuracy: 0.9924
241/375 [==================>...........] - ETA: 8s - loss: 0.0243 - accuracy: 0.9925
242/375 [==================>...........] - ETA: 8s - loss: 0.0243 - accuracy: 0.9925
243/375 [==================>...........] - ETA: 8s - loss: 0.0242 - accuracy: 0.9925
244/375 [==================>...........] - ETA: 8s - loss: 0.0241 - accuracy: 0.9926
245/375 [==================>...........] - ETA: 7s - loss: 0.0240 - accuracy: 0.9926
246/375 [==================>...........] - ETA: 7s - loss: 0.0239 - accuracy: 0.9926
247/375 [==================>...........] - ETA: 7s - loss: 0.0238 - accuracy: 0.9927
248/375 [==================>...........] - ETA: 7s - loss: 0.0239 - accuracy: 0.9927
249/375 [==================>...........] - ETA: 7s - loss: 0.0239 - accuracy: 0.9926
250/375 [===================>..........] - ETA: 7s - loss: 0.0239 - accuracy: 0.9925
251/375 [===================>..........] - ETA: 7s - loss: 0.0241 - accuracy: 0.9925
252/375 [===================>..........] - ETA: 7s - loss: 0.0241 - accuracy: 0.9925
253/375 [===================>..........] - ETA: 7s - loss: 0.0243 - accuracy: 0.9925
254/375 [===================>..........] - ETA: 7s - loss: 0.0242 - accuracy: 0.9925
255/375 [===================>..........] - ETA: 7s - loss: 0.0241 - accuracy: 0.9925
256/375 [===================>..........] - ETA: 7s - loss: 0.0241 - accuracy: 0.9925
257/375 [===================>..........] - ETA: 7s - loss: 0.0241 - accuracy: 0.9925
258/375 [===================>..........] - ETA: 7s - loss: 0.0240 - accuracy: 0.9926
259/375 [===================>..........] - ETA: 7s - loss: 0.0239 - accuracy: 0.9926
260/375 [===================>..........] - ETA: 7s - loss: 0.0239 - accuracy: 0.9926
261/375 [===================>..........] - ETA: 7s - loss: 0.0238 - accuracy: 0.9926
262/375 [===================>..........] - ETA: 7s - loss: 0.0238 - accuracy: 0.9926
263/375 [====================>.........] - ETA: 6s - loss: 0.0237 - accuracy: 0.9927
264/375 [====================>.........] - ETA: 6s - loss: 0.0236 - accuracy: 0.9927
265/375 [====================>.........] - ETA: 6s - loss: 0.0235 - accuracy: 0.9927
266/375 [====================>.........] - ETA: 6s - loss: 0.0235 - accuracy: 0.9927
267/375 [====================>.........] - ETA: 6s - loss: 0.0235 - accuracy: 0.9927
268/375 [====================>.........] - ETA: 6s - loss: 0.0235 - accuracy: 0.9927
269/375 [====================>.........] - ETA: 6s - loss: 0.0235 - accuracy: 0.9927
270/375 [====================>.........] - ETA: 6s - loss: 0.0235 - accuracy: 0.9927
271/375 [====================>.........] - ETA: 6s - loss: 0.0234 - accuracy: 0.9927
272/375 [====================>.........] - ETA: 6s - loss: 0.0233 - accuracy: 0.9927
273/375 [====================>.........] - ETA: 6s - loss: 0.0233 - accuracy: 0.9927
274/375 [====================>.........] - ETA: 6s - loss: 0.0232 - accuracy: 0.9927
276/375 [=====================>........] - ETA: 6s - loss: 0.0231 - accuracy: 0.9928
277/375 [=====================>........] - ETA: 6s - loss: 0.0231 - accuracy: 0.9928
279/375 [=====================>........] - ETA: 5s - loss: 0.0230 - accuracy: 0.9928
280/375 [=====================>........] - ETA: 5s - loss: 0.0231 - accuracy: 0.9928
282/375 [=====================>........] - ETA: 5s - loss: 0.0238 - accuracy: 0.9927
283/375 [=====================>........] - ETA: 5s - loss: 0.0238 - accuracy: 0.9927
284/375 [=====================>........] - ETA: 5s - loss: 0.0238 - accuracy: 0.9927
285/375 [=====================>........] - ETA: 5s - loss: 0.0238 - accuracy: 0.9927
286/375 [=====================>........] - ETA: 5s - loss: 0.0239 - accuracy: 0.9926
287/375 [=====================>........] - ETA: 5s - loss: 0.0239 - accuracy: 0.9926
288/375 [======================>.......] - ETA: 5s - loss: 0.0238 - accuracy: 0.9926
289/375 [======================>.......] - ETA: 5s - loss: 0.0237 - accuracy: 0.9927
290/375 [======================>.......] - ETA: 5s - loss: 0.0237 - accuracy: 0.9927
291/375 [======================>.......] - ETA: 5s - loss: 0.0237 - accuracy: 0.9927
292/375 [======================>.......] - ETA: 5s - loss: 0.0236 - accuracy: 0.9927
293/375 [======================>.......] - ETA: 5s - loss: 0.0235 - accuracy: 0.9927
294/375 [======================>.......] - ETA: 5s - loss: 0.0235 - accuracy: 0.9928
295/375 [======================>.......] - ETA: 5s - loss: 0.0234 - accuracy: 0.9928
296/375 [======================>.......] - ETA: 4s - loss: 0.0236 - accuracy: 0.9928
297/375 [======================>.......] - ETA: 4s - loss: 0.0236 - accuracy: 0.9928
298/375 [======================>.......] - ETA: 4s - loss: 0.0236 - accuracy: 0.9928
299/375 [======================>.......] - ETA: 4s - loss: 0.0236 - accuracy: 0.9928
300/375 [=======================>......] - ETA: 4s - loss: 0.0236 - accuracy: 0.9928
301/375 [=======================>......] - ETA: 4s - loss: 0.0235 - accuracy: 0.9928
302/375 [=======================>......] - ETA: 4s - loss: 0.0236 - accuracy: 0.9928
304/375 [=======================>......] - ETA: 4s - loss: 0.0235 - accuracy: 0.9928
305/375 [=======================>......] - ETA: 4s - loss: 0.0235 - accuracy: 0.9928
306/375 [=======================>......] - ETA: 4s - loss: 0.0235 - accuracy: 0.9928
308/375 [=======================>......] - ETA: 4s - loss: 0.0236 - accuracy: 0.9928
310/375 [=======================>......] - ETA: 4s - loss: 0.0235 - accuracy: 0.9928
312/375 [=======================>......] - ETA: 3s - loss: 0.0236 - accuracy: 0.9928
313/375 [========================>.....] - ETA: 3s - loss: 0.0235 - accuracy: 0.9928
314/375 [========================>.....] - ETA: 3s - loss: 0.0235 - accuracy: 0.9928
315/375 [========================>.....] - ETA: 3s - loss: 0.0235 - accuracy: 0.9928
316/375 [========================>.....] - ETA: 3s - loss: 0.0235 - accuracy: 0.9928
317/375 [========================>.....] - ETA: 3s - loss: 0.0234 - accuracy: 0.9928
318/375 [========================>.....] - ETA: 3s - loss: 0.0235 - accuracy: 0.9928
320/375 [========================>.....] - ETA: 3s - loss: 0.0235 - accuracy: 0.9928
321/375 [========================>.....] - ETA: 3s - loss: 0.0235 - accuracy: 0.9928
322/375 [========================>.....] - ETA: 3s - loss: 0.0234 - accuracy: 0.9928
324/375 [========================>.....] - ETA: 3s - loss: 0.0235 - accuracy: 0.9928
325/375 [=========================>....] - ETA: 3s - loss: 0.0237 - accuracy: 0.9928
326/375 [=========================>....] - ETA: 3s - loss: 0.0237 - accuracy: 0.9928
327/375 [=========================>....] - ETA: 3s - loss: 0.0237 - accuracy: 0.9928
328/375 [=========================>....] - ETA: 2s - loss: 0.0237 - accuracy: 0.9928
329/375 [=========================>....] - ETA: 2s - loss: 0.0237 - accuracy: 0.9927
330/375 [=========================>....] - ETA: 2s - loss: 0.0237 - accuracy: 0.9927
331/375 [=========================>....] - ETA: 2s - loss: 0.0237 - accuracy: 0.9927
332/375 [=========================>....] - ETA: 2s - loss: 0.0237 - accuracy: 0.9928
333/375 [=========================>....] - ETA: 2s - loss: 0.0237 - accuracy: 0.9928
334/375 [=========================>....] - ETA: 2s - loss: 0.0237 - accuracy: 0.9928
335/375 [=========================>....] - ETA: 2s - loss: 0.0237 - accuracy: 0.9927
336/375 [=========================>....] - ETA: 2s - loss: 0.0237 - accuracy: 0.9928
337/375 [=========================>....] - ETA: 2s - loss: 0.0236 - accuracy: 0.9928
338/375 [==========================>...] - ETA: 2s - loss: 0.0236 - accuracy: 0.9928
339/375 [==========================>...] - ETA: 2s - loss: 0.0237 - accuracy: 0.9928
340/375 [==========================>...] - ETA: 2s - loss: 0.0237 - accuracy: 0.9928
341/375 [==========================>...] - ETA: 2s - loss: 0.0236 - accuracy: 0.9928
342/375 [==========================>...] - ETA: 2s - loss: 0.0236 - accuracy: 0.9928
343/375 [==========================>...] - ETA: 2s - loss: 0.0236 - accuracy: 0.9928
344/375 [==========================>...] - ETA: 1s - loss: 0.0235 - accuracy: 0.9929
345/375 [==========================>...] - ETA: 1s - loss: 0.0235 - accuracy: 0.9929
346/375 [==========================>...] - ETA: 1s - loss: 0.0236 - accuracy: 0.9929
347/375 [==========================>...] - ETA: 1s - loss: 0.0235 - accuracy: 0.9929
348/375 [==========================>...] - ETA: 1s - loss: 0.0235 - accuracy: 0.9929
349/375 [==========================>...] - ETA: 1s - loss: 0.0235 - accuracy: 0.9929
350/375 [===========================>..] - ETA: 1s - loss: 0.0234 - accuracy: 0.9929
351/375 [===========================>..] - ETA: 1s - loss: 0.0234 - accuracy: 0.9929
352/375 [===========================>..] - ETA: 1s - loss: 0.0234 - accuracy: 0.9929
353/375 [===========================>..] - ETA: 1s - loss: 0.0236 - accuracy: 0.9929
354/375 [===========================>..] - ETA: 1s - loss: 0.0235 - accuracy: 0.9930
355/375 [===========================>..] - ETA: 1s - loss: 0.0236 - accuracy: 0.9929
356/375 [===========================>..] - ETA: 1s - loss: 0.0235 - accuracy: 0.9929
357/375 [===========================>..] - ETA: 1s - loss: 0.0236 - accuracy: 0.9929
358/375 [===========================>..] - ETA: 1s - loss: 0.0235 - accuracy: 0.9930
359/375 [===========================>..] - ETA: 1s - loss: 0.0235 - accuracy: 0.9929
360/375 [===========================>..] - ETA: 0s - loss: 0.0235 - accuracy: 0.9929
361/375 [===========================>..] - ETA: 0s - loss: 0.0235 - accuracy: 0.9929
362/375 [===========================>..] - ETA: 0s - loss: 0.0235 - accuracy: 0.9929
363/375 [============================>.] - ETA: 0s - loss: 0.0236 - accuracy: 0.9929
364/375 [============================>.] - ETA: 0s - loss: 0.0237 - accuracy: 0.9929
365/375 [============================>.] - ETA: 0s - loss: 0.0236 - accuracy: 0.9929
366/375 [============================>.] - ETA: 0s - loss: 0.0237 - accuracy: 0.9929
367/375 [============================>.] - ETA: 0s - loss: 0.0236 - accuracy: 0.9929
368/375 [============================>.] - ETA: 0s - loss: 0.0236 - accuracy: 0.9929
369/375 [============================>.] - ETA: 0s - loss: 0.0235 - accuracy: 0.9929
370/375 [============================>.] - ETA: 0s - loss: 0.0235 - accuracy: 0.9929
371/375 [============================>.] - ETA: 0s - loss: 0.0234 - accuracy: 0.9930
372/375 [============================>.] - ETA: 0s - loss: 0.0234 - accuracy: 0.9930
373/375 [============================>.] - ETA: 0s - loss: 0.0233 - accuracy: 0.9930
374/375 [============================>.] - ETA: 0s - loss: 0.0233 - accuracy: 0.9930
375/375 [==============================] - 24s 65ms/step - loss: 0.0235 - accuracy: 0.9930

375/375 [==============================] - 26s 70ms/step - loss: 0.0235 - accuracy: 0.9930 - val_loss: 0.0479 - val_accuracy: 0.9876

history

Final epoch (plot to see history):
        loss: 0.02349
    accuracy: 0.993
    val_loss: 0.04794
val_accuracy: 0.9876 

plot(history)

Evaluation

Using our CNN we obtain a test set accuracy of ~ 0.99.

model %>% evaluate(test_images, test_labels, verbose = FALSE)

      loss   accuracy 
0.03527408 0.99000001 

Your Turn! (10 min)

Spend 10 minutes adjusting various CNN components:

• Change the number of filters
• Change filter/kernel size
• Adjust the stride
• Add padding
• Add more convolution layers

Or keep the same CNN components as above but apply some of the tuning steps we covered this morning:

• Try different adaptive learning rate optimizers and learning rate values
• How does batch size impact performance
• You can even try to add weight decay or dropout to each layer to control overfitting:
  • weight decays can be applied with kernel_regularizer within layer_conv_2d
  • layer_dropout() can be applied before or after pooling but is more commonly seen after. Note that dropout in CNNs will randomly drop out entire feature maps

model <- keras_model_sequential() %>%
  
  layer_conv_2d(filters = ____, kernel_size = ____, activation = "relu", 
                input_shape = c(28, 28, 1)) %>%
  layer_max_pooling_2d(pool_size = ____) %>%
  
  layer_conv_2d(filters = ____, kernel_size = ____, activation = "relu") %>%
  layer_max_pooling_2d(pool_size = ____) %>%
  
  layer_conv_2d(filters = ____, kernel_size = ____, activation = "relu")

model %>%
  _______ %>%
  layer_dense(units = 64, activation = "relu") %>%
  layer_dense(units = 10, activation = "softmax")

model %>% compile(
  optimizer = "rmsprop",
  loss = "categorical_crossentropy",
  metrics = c("accuracy")
)

summary(model)

history <- model %>% fit(
  train_images, train_labels,
  epochs = 5, 
  batch_size = 64,
  validation_split = 0.2
)

Key takeaways

• CNNs allow us to capture and control for image variance

• The convolution layer provides the main mechanism for feature engineering

  • We slide a filter/kernel over our images to create feature maps
  • We can use striding and padding to control the size of our feature maps
  • Apply pooling to downsample

• Always flatten the output of the convolution layer to feed into a dense layer

• Since all our feature engineering occurs in the convolution layer, we typically need only one hidden dense layer with far fewer units and epochs to train our model

🏠